What
quire validate has no --json. --diagnostics-format json emits NDJSON, but every locatable fact is embedded in one string:
{"kind":"ValidationError","message":"spec/tests.md: line 10: [TestMatrix] 'test_cases': TC-001: column 'Type' cell 'Demonstration' is not one of [\"Unit\", \"Integration\", \"Inspection\"] [assert]","severity":"error"}The path, the line, the archetype, the declaration key, the row id and the reason are all there — and all of them are only available by parsing prose. coverage --json publishes the same class of fact as fields ({"document": "...", "row_id": "TC-001", "line": 10}), so the two commands disagree about whether a finding is data.
Why it matters
agent-ix/quoin's tier-1 benchmark (quoin#199) must map findings to defect families by reason, and locate them by path:line, to compute finding_precision, finding_recall and finding_localisation_rate. For every validate-sourced family it does that with:
/^(?<path>.+?): line (?<line>\d+): (?<rest>.*) \[(?<reason>[a-z-]+)\]$/
A message-format change silently stops those families scoring — and a family that stops scoring reads exactly like a family with nothing to report, which is the failure mode that whole programme exists to end. The runner currently refuses to skip a message it cannot parse (it throws), but refusing loudly is a workaround, not a contract.
It is the same argument coverage --json already won: a consumer that has to regex your prose is a consumer you have not given an interface to.
Proposed
Give --diagnostics-format json structured fields alongside message, rather than instead of it:
{
"kind": "ValidationError",
"severity": "error",
"reason": "assert",
"path": "spec/tests.md",
"line": 10,
"archetype": "TestMatrix",
"declaration": "test_cases",
"row_id": "TC-001",
"message": "<unchanged human string>"
}message stays byte-identical so no human surface changes and no existing consumer breaks. The engine already holds every one of these as a typed value — ValidationError carries message, line and reason, and AssertFailure carries row_id — so this is a serialization change at the boundary, not new analysis.
Related
agent-ix/quire-rs#254 — the line this would publish was wrong until that fix (row-scoped asserts named the separator, not the row). Structured output makes that class of defect visible to a consumer instead of invisible inside a string.agent-ix/quoin#199 — the tier-1 runner and the regex above; bench/tier1-mapping.json records this as parse_fragility and points here.
What
quire validatehas no--json.--diagnostics-format jsonemits NDJSON, but every locatable fact is embedded in one string:{"kind":"ValidationError","message":"spec/tests.md: line 10: [TestMatrix] 'test_cases': TC-001: column 'Type' cell 'Demonstration' is not one of [\"Unit\", \"Integration\", \"Inspection\"] [assert]","severity":"error"}The path, the line, the archetype, the declaration key, the row id and the reason are all there — and all of them are only available by parsing prose.
coverage --jsonpublishes the same class of fact as fields ({"document": "...", "row_id": "TC-001", "line": 10}), so the two commands disagree about whether a finding is data.Why it matters
agent-ix/quoin's tier-1 benchmark (quoin#199) must map findings to defect families by reason, and locate them bypath:line, to computefinding_precision,finding_recallandfinding_localisation_rate. For everyvalidate-sourced family it does that with:/^(?<path>.+?): line (?<line>\d+): (?<rest>.*) \[(?<reason>[a-z-]+)\]$/A message-format change silently stops those families scoring — and a family that stops scoring reads exactly like a family with nothing to report, which is the failure mode that whole programme exists to end. The runner currently refuses to skip a message it cannot parse (it throws), but refusing loudly is a workaround, not a contract.
It is the same argument
coverage --jsonalready won: a consumer that has to regex your prose is a consumer you have not given an interface to.Proposed
Give
--diagnostics-format jsonstructured fields alongsidemessage, rather than instead of it:{ "kind": "ValidationError", "severity": "error", "reason": "assert", "path": "spec/tests.md", "line": 10, "archetype": "TestMatrix", "declaration": "test_cases", "row_id": "TC-001", "message": "<unchanged human string>" }messagestays byte-identical so no human surface changes and no existing consumer breaks. The engine already holds every one of these as a typed value —ValidationErrorcarriesmessage,lineandreason, andAssertFailurecarriesrow_id— so this is a serialization change at the boundary, not new analysis.Related
agent-ix/quire-rs#254— thelinethis would publish was wrong until that fix (row-scoped asserts named the separator, not the row). Structured output makes that class of defect visible to a consumer instead of invisible inside a string.agent-ix/quoin#199— the tier-1 runner and the regex above;bench/tier1-mapping.jsonrecords this asparse_fragilityand points here.