The six mechanically checkable criterion types

Deterministic, no AI judging. Each example below is one criterion you can paste into criteria.

  1. evidence_present — the item exists and is non-empty. A missing key, null, an empty array and a whitespace-only string all fail.
{"type":"evidence_present","key":"test_log"}
  1. hash_matches — sha256 of the value's UTF-8 bytes equals the declared digest. A string value is hashed as-is, matching sha256sum <file>; a non-string value is hashed as its compact JSON.stringify output.
{"type":"hash_matches","path":"changelog","sha256":"3bfc269594ef649228e9a74bab00f042efc91d5acc6fbee31a382e80d42388fe"}
  1. url_status — the URL returns the expected integer status code.
{"type":"url_status","url":"https://example.com","expect":200}
  1. url_contains — the fetched content contains the string (case-sensitive).
{"type":"url_contains","url":"https://example.com","contains":"Example"}
  1. json_path — a dotted path into evidence exists; with equals, the two are compared by string form, so 9 matches "9".
{"type":"json_path","path":"test_summary.total","equals":9}
  1. count_min — an array's length, an object's key count, or the number itself is at least min.
{"type":"count_min","path":"boundary_cases","min":3}

key and path both accept dotted paths (a.b.c). Both URL types also accept render: true. When a key is missing, missing names the top-level evidence keys you actually sent, so one retry is usually enough.


All questions · nodcheck · Source: https://nodcheck.com/answers/six-check-types