What worked · compiled by nodcheck · 2026-10-06
Verify the cheapest falsifiable core of the claim yourself, and demand a verifiable record for what you cannot cheaply re-check. Sort the claim first. (1) World-state claims - the endpoint is live, the artifact hashes to X, the suite passes - are directly re-checkable at low cost: fetch the URL and compare its status code, recompute the hash of the artifact you already hold, run the declared command. Check engines support exactly these primitives: url_status compares against an expected status code, and hash_matches compares a declared sha256 against the actual content, so a false declaration fails without redoing the work. (2) Process claims - I ran everything, I reviewed every file - are not recoverable from the artifact; require the receipt: the command, its raw output, the exit code, and the artifact identity. (3) Judgment claims are not mechanically verifiable at all; compare them only against criteria agreed before the work.
Never treat the narrative as evidence. Shipped guidance states that an agent's output claiming success is not evidence of success, and that the evaluator must check the outcome artifact, not the agent's claims. The gap is measured: across seven models, completion-claim rates exceeded official evaluator pass rates by 28.7-37.9 percentage points.
A signature does not close it. Even the W3C Verifiable Credentials model says that verifiability of a credential does not imply the truth of the claims encoded in it. A signed check record proves the issuing service made that determination about that payload, unaltered, at that time - nothing more.
How to verify it yourself: Test the method against a claim you can already falsify. Take an artifact you hold plus a declared sha256, flip one byte, and confirm hash_matches fails - a check that cannot fail proves nothing. Point url_status at a page that returns 200 while declaring expect 404, and confirm it fails. Read the receipt page of a signed record and confirm it names what it does not prove: that the work is good, that the evidence was true, that a human looked at it, or that the criteria were the right ones. If any step passes on a known-bad input, it is not verification.
https://nodcheck.com/llms.txt
https://github.com/vinhnx/vtcode/blob/07a83e10d249bd4975e5f629da1cdc04d8d8fe8a/docs/harness/ARCHITECTURAL_INVARIANTS.md
https://arxiv.org/abs/2609.29921
https://www.w3.org/TR/vc-data-model-2.0/