Skip to content

fix(verify): preserve structural claim failure - #596

Draft
altrudev wants to merge 1 commit into
agentrust-io:mainfrom
altrudev:fix/verify-malformed-claims-592
Draft

fix(verify): preserve structural claim failure#596
altrudev wants to merge 1 commit into
agentrust-io:mainfrom
altrudev:fix/verify-malformed-claims-592

Conversation

@altrudev

@altrudev altrudev commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Closes #592.

Maintainer ruling and correction

Reconstructed on current upstream main after the review of 141c5b2. The strict schema prerequisite is preserved: any RuntimeClaim validation failure returns CLAIM_MALFORMED before signature, key-binding or other downstream interpretation. No general ordering between signature and key-binding failures is introduced.

The intermediate compatibility workaround at 877c506 was too broad and has been removed. The original strict structural boundary and its 15 regression tests are preserved byte-for-byte. The positive-path TPM fixtures now use the valid, signed gateway.attestation_evidence envelope rather than placing evidence in schema-closed trace.runtime.

Fixture and representation corrections

  • TPM raw evidence is supplied through the producer's AttestationReportInfo and included in the final envelope signature.
  • The separately supplied EK certificate chain is passed through the producer fixture and signed with the rest of the claim, preserving the intended EK-chain verification assertion.
  • A malformed measurement is now expected to stop at the schema boundary, rather than reaching the TPM parser.
  • The cross-platform evidence test retains positive coverage for supported Azure CVM, AMD SEV-SNP and Intel TDX platforms. The old OPAQUE positive case used a platform value not permitted by TRACE 0.10.0. It is now an explicit refusal regression, along with legacy unsupported aliases; no schema expansion or alias promotion is proposed.

DDC executable evidence

Exact candidate 6ffad28da75560dfba5a0c242839403f871e9c53, parent cc27f91ec9b2d623dc633e24f1c368fb20f7566f (current upstream main at reconstruction). One DCO-signed commit, five changed files, no unrelated source changes. Tested in an isolated local virtual environment with Python 3.14.4, agent-manifest 0.12.0 and agentrust-trace 0.10.0.

  • ruff check src/ tests/: PASS.
  • mypy src/cmcp_runtime/: PASS, 54 source files.
  • mypy src/cmcp_verify/verify.py: PASS.
  • Focused structural/TPM/cross-platform matrix: 64 passed.
  • Full python -m pytest -q: 1,671 passed, 14 skipped.
  • CI-scoped unit/conformance/integration suite with coverage: 1,629 passed, 6 skipped; coverage 87.56%, above the required 70%.
  • Original strict schema implementation and all 15 new regression tests were independently compared against 141c5b2 and found byte-for-byte identical.
  • The earlier 25-failure candidate and its 5 TPM regressions are superseded; the current candidate has no locally observed test failures. Hosted checks on the replacement head are not yet claimed.

Scope and lineage

No schema, wire-format, cryptographic primitive, or general failure-precedence change. The old draft history is replaced by a single signed-off commit on current main, preserving original implementation credit and making the DCO history complete. The PR remains draft for maintainer review and hosted validation.

AI-assistance disclosure: ChatGPT assisted with source triage, DDC boundary analysis, implementation and fixture correction, local differential testing, clean-history reconstruction, and exact-diff review. altrudev reviewed the bounded claim and remains responsible for the contribution.

@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

🟡 Contributor Check: MEDIUM

Check Result
Profile MEDIUM
Credential LOW
Overall MEDIUM

Automated check by AgenTrust Contributor Check.

@github-actions github-actions Bot added the needs-review:MEDIUM Contributor check flagged MEDIUM risk label Aug 30, 2026
@imran-siddique

imran-siddique commented Aug 31, 2026

Copy link
Copy Markdown
Member

Batch response for this cluster is here: agentrust-io/agent-manifest#357 (comment)

Short version: the finding class is real and welcome. Your CI had never run, held under first-time-contributor gating, until I released 36 runs across your PRs an hour ago, and five of your eight are now red. Please fix those, sequence trace-spec#258 against #252 which touch the same two files, and tell me the order you want them reviewed in.

Copy link
Copy Markdown
Contributor Author

I reconciled the released CI failure. The red step was ruff import formatting in the new test file, before the reproducer could execute; that is fixed at 4e2cce1.

Because this PR is explicitly a draft test-first reproducer, I also changed the current-head behavior at 99ba8a3 so only the known host-language escape (AttributeError / TypeError) is reported with pytest.xfail. If verify_trace_claim() returns normally, the original VerificationResult and schema assertions run unchanged. This keeps a known-current defect from making the whole draft CI red without weakening the desired invariant or silently choosing the still-open failure-precedence rule.

I am keeping #596 out of the review queue until the implementation/precedence decision is ready. Fresh CI is currently waiting on workflow approval; no green claim yet.

@imran-siddique

Copy link
Copy Markdown
Member

Thanks for splitting this the way you did. Asserting only the uncontested invariant and holding the precedence question for a maintainer is the right call, and it is the reason this is a useful reproducer rather than a change that quietly decides something.

The ruling: structural malformation wins.

If the claim structure is malformed, CLAIM_MALFORMED is the result, and a subsequent signature or key-binding failure must not overwrite it. Encode that as first-failure-wins for the structural stage specifically, not as a general ordering over all failure kinds.

The reason is not aesthetic. A signature verdict is a statement about a specific set of bytes. When the structure did not parse, the verifier never established what those bytes were, so whatever it computed is a verdict about something it could not identify. Reporting it as SIGNATURE_INVALID tells the caller the signature was checked and found bad. It was not checked. That is the same defect class I have been unpicking across the other repos: absence of evidence surfacing as a negative verdict rather than as "not established". A malformed record is not a record with a bad signature; it is a record whose signature is unevaluated.

The practical consequence is the one that matters to a caller: SIGNATURE_INVALID says do not trust this issuer, and CLAIM_MALFORMED says do not trust this input. Those route to different places, and collapsing the second into the first sends operators after the wrong party.

Two things I do want from the shape of it:

  1. Keep the malformed result carrying which intermediate failed to parse. Your matrix already covers trace, cnf, jwk, jwk.x, gateway, audit_chain and attestation_evidence; a caller debugging a producer needs to know which one, and a single flat CLAIM_MALFORMED throws that away.
  2. Do not extend the precedence rule past the structural stage on my say-so. Ordering between signature failure and key-binding failure is a separate question with its own arguments, and this PR does not need to answer it.

Once that is in, I am happy for the xfails on the escaping shapes to become plain assertions, since the escape itself is a bug we agree on.

Separately, on CI: the one thing still red is ruff rule I001 on the import block in the new test file, at head 99ba8a3. It reports as test (3.13, ubuntu-latest) but the failing step is Lint, so nothing in your matrix has executed yet. ruff check --fix on that file clears it.

@altrudev
altrudev force-pushed the fix/verify-malformed-claims-592 branch from 99ba8a3 to 141c5b2 Compare September 2, 2026 18:03
@altrudev altrudev changed the title test(verify): reproduce malformed TRACE Claim shape escapes fix(verify): preserve structural claim failure Sep 2, 2026

@imran-siddique imran-siddique left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review/test follow-up on 141c5b2ee96c.

The schema-first return correctly prevents malformed claims from reaching cryptographic interpretation, and the new boundary regressions pass. The full suite is not green: fresh CI reports 25 failed, 1470 passed, 6 skipped. Besides the shared SDK/TPM-root failures tracked by #616, five existing TPM tests now stop at schema validation because their claims contain trace.runtime.raw_evidence, which RuntimeClaim rejects. Update those positive-path fixtures to the valid claim/evidence shape and verify that their intended TPM checks still execute; preserve the early CLAIM_MALFORMED return. Local results also include a distribution metadata mismatch and an oversized-request test failure. Keep this draft until the full suite passes.

Full local python -m pytest -q (Windows, Python 3.12, existing review environment): 7 failed, 1528 passed, 16 skipped, 1 warning in 64.20s (0:01:04).

Fresh workflow evidence: 33664858111 (failure).

@codecov-commenter

codecov-commenter commented Sep 6, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 75.00000% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/cmcp_verify/verify.py 75.00% 5 Missing ⚠️

📢 Thoughts on this report? Let us know!

Closes agentrust-io#592. Reconstruct agentrust-io#596 on current main with the strict schema prerequisite preserved. Move legacy TPM fixtures to the signed gateway evidence envelope and pin schema refusal for unsupported platform aliases. Supersedes the earlier draft implementation and its overbroad compatibility workaround.

AI-assisted implementation and verification; Altru.dev remains responsible for the contribution.

Signed-off-by: Altru.dev <altrudevelop@gmail.com>
@altrudev
altrudev force-pushed the fix/verify-malformed-claims-592 branch from 877c506 to 6ffad28 Compare September 6, 2026 23:30

altrudev commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

@imran-siddique Thank you for the review. I have replaced the intermediate compatibility workaround with the strict schema-first implementation you requested and corrected the positive-path fixtures instead. The new head is 6ffad28da75560dfba5a0c242839403f871e9c53, reconstructed on current main cc27f91 as one DCO-signed commit.

The original structural boundary and 15 regression tests are preserved byte-for-byte. TPM raw evidence and the separate EK chain now travel through the signed gateway envelope. The full local suite is green (1,671 passed, 14 skipped), as is the CI-scoped suite (1,629 passed, 6 skipped; 87.56% coverage), Ruff, and mypy. I also corrected the cross-platform fixture that had treated a TRACE-unsupported OPAQUE platform value as a valid positive case; supported hardware paths retain positive coverage and unsupported aliases now have explicit schema-refusal tests.

The PR body contains the exact test evidence and scope. I am keeping it draft until your review and hosted checks are complete. No manual Actions run was triggered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:MEDIUM Contributor check flagged MEDIUM risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

verify_trace_claim can raise TypeError after schema failure on malformed nested claim shapes

3 participants