Skip to content

docs(rfc): propose runtime evidence, and what a verifier may conclude without it - #277

Open
imran-siddique wants to merge 4 commits into
mainfrom
rfc/runtime-evidence-v03
Open

docs(rfc): propose runtime evidence, and what a verifier may conclude without it#277
imran-siddique wants to merge 4 commits into
mainfrom
rfc/runtime-evidence-v03

Conversation

@imran-siddique

Copy link
Copy Markdown
Member

The gap

A Trust Record states a hardware platform and a measurement, and carries nothing a relying party can check either against. runtime is additionalProperties: false over five members, none of which is the quote, the certificate chain, or the report signature. appraisal is a status enum plus a verifier URI, which is a pointer to a party who says they checked.

verify_record() performs no attestation verification, and the package contains none: the string quote appears in src/agentrust_trace/ only inside comments. That is not an implementation gap. There is nothing in the record for such code to consume.

So the chain a relying party actually gets is: the record is signed, the signature is sound, and it attests authorship. Every hardware claim inside is the issuer's word, and a verifier reporting "TEE-attested" on that basis is repeating the issuer's assertion in the verifier's voice.

Why this is our own doctrine, not a new idea

The spec already reasons about assurance laundering, carefully, and in one direction only.

§3.1.1 forces software-only onto an assembled record because "an importer holding someone else's log has no quote to present". §3.1.2 rule 3 forbids treating a resolved references entry as attested evidence, naming assurance laundering as the reason. Both govern pointers to things outside the record. Neither governs runtime, where the same move is not merely possible but is all that is on offer: rim_uri is a pointer, measurement is a transcription, appraisal.verifier names someone else's verdict.

It is also a requirement already made and never made checkable. docs/trust-levels.md:50 requires a Level 1 signing key to be generated inside a verified TEE, and spec/trace-v0.2.md §5 maps runtime to "RATS Evidence + vendor RIM". Nothing represents either. The requirement, the mapping, and the schema disagree, and the schema is what runs.

What this adds

  • runtime.evidence carrying the quote, in schema/trace-claim-v0.3-draft.json, generated from the v0.2 schema so the diff is exactly one member deep.
  • Seven verification rules, envelope-first, so the record signature binds a quote to this record rather than to any record.
  • Three grades rather than a boolean: unattested, platform-attested, attested. The middle one was forced by the artifacts, not chosen.
  • §6.1: a grade on the record does not propagate to the claims inside it.

Additive. No existing field, schema, or record changes. A record without evidence grades unattested, which is a name for what every v0.2 record already was.

Conformance material

Nine vectors over two genuine Intel TDX v4 quotes from a GCP C3 guest, verified by agent-manifest's verifier imported unmodified. Nothing is minted. TRACE does not implement attestation and this proposal does not start.

9/9 vectors behaved as the profile says they must (1 of them documenting a limit).

Three findings, all kept rather than smoothed

The substitution vector was written to be a rejection and is not one. Both captures come from one TD, so they share an MRTD, and the measurement rule cannot separate two quotes that agree on the measurement. The rule binds a record to a measurement, never to a quote. That caps the middle grade at "genuine silicon reporting this measurement" and is the argument for the top grade existing. A synthetic corpus would have minted different measurements and passed.

The top grade is currently unreachable. No capture we hold binds a record-signing key, and the pre-image behind the bindings we do have was never committed, so REPORT_DATA is a value that verifiably came from the TEE and cannot be opened. Both are capture-procedure gaps, cheap to close on the next TDX run, and cheaper to say now than to discover during adoption.

The first draft of §6.1 contained the bug §6.1 forbids. It graded the model claim by reading evidence.binds, which would let a producer raise its own claim by writing a string. §3 already said a verifier recomputes binds and never reads it; the implementation forty lines away did the opposite. Same shape as the gap in runtime this whole proposal is about. Fixed in the second commit, with a vector that declares a binding it does not have and stays self-reported.

Status

Informative, lowercase keywords, per CONTRIBUTING.md. Binds nothing. It does not resolve §5.2: the reference producer binds a manifest digest rather than the signing key today, and a profile is not adopted by writing down which side should move.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LKXk6sx5CzAsHmTEJhk7Gy

imran-siddique and others added 2 commits September 3, 2026 13:41
… without it

A Trust Record states a hardware platform and a measurement, and carries nothing a
relying party can check either against. `runtime` is additionalProperties:false over
five members, none of which is the quote, the certificate chain, or the report
signature. `verify_record()` performs no attestation verification, and the package
contains none: "quote" appears in src/agentrust_trace/ only inside comments. That is
not an implementation gap, because there is nothing in the record for such code to
consume.

The specification already reasons about this, in one direction only. 3.1.1 forces
software-only on an assembled record because "an importer holding someone else's log
has no quote to present". 3.1.2 rule 3 forbids treating a resolved reference as
attested evidence, naming assurance laundering as the reason. Both govern pointers to
things outside the record. Neither governs `runtime`, where the same move is not just
possible but is all that is on offer: rim_uri is a pointer, measurement is a
transcription, appraisal.verifier names someone else's verdict. This proposal is the
project's own doctrine applied to the block it was never applied to.

It is also a requirement already made and never made checkable. docs/trust-levels.md
line 50 requires a Level 1 signing key to be generated inside a verified TEE, and
spec 5 maps `runtime` to "RATS Evidence + vendor RIM". Nothing represents either. The
requirement, the mapping and the schema disagree, and the schema is what runs.

Adds a `runtime.evidence` member (draft schema generated from the v0.2 one, so the diff
is one member deep), seven verification rules, and three grades: unattested,
platform-attested, attested. The middle grade was forced by the artifacts rather than
chosen, and 6.1 keeps a grade on the record from propagating to the claims inside it,
which is the same anti-laundering rule turned inward.

The corpus is eight vectors over two GENUINE Intel TDX v4 quotes from a GCP C3 guest,
verified by agent-manifest's verifier imported unmodified. TRACE does not implement
attestation and this does not start.

Two findings, both kept rather than smoothed:

  - The substitution vector was written to be a rejection and is not one. Both captures
    come from one TD, so they share an MRTD, and the measurement rule cannot separate
    two quotes that agree on the measurement. The rule binds a record to a measurement
    and never to a quote. That caps the middle grade at "genuine silicon reporting this
    measurement" and is the argument for the top grade existing. A synthetic corpus
    would have minted different measurements and passed.

  - The top grade is currently unreachable. No capture we hold binds a record-signing
    key, and the pre-image behind the bindings we do have was never committed, so
    REPORT_DATA is a value that verifiably came from the TEE and cannot be opened. Both
    are gaps in the capture procedure, cheap to close on the next run, and cheaper to
    say now than to discover during adoption.

Informative throughout, lowercase keywords, per CONTRIBUTING.md. Binds nothing. No
existing field, schema or record changes; a record without `evidence` grades
unattested, which is a name for what every v0.2 record already was.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKXk6sx5CzAsHmTEJhk7Gy
Two defects, one of them the kind this proposal is about.

The schema enum is `weights-digest` and `grade_model_claim` compared against
`weights_digest`, so the branch could not fire. No vector set `binds`, so nothing
noticed.

Correcting the spelling would have made it worse. Reading `binds` at all lets a
producer raise its own model claim by writing a string into a record, which is the
assurance laundering section 6.1 exists to forbid, reintroduced inside the section
that forbids it. Section 3 already said a verifier recomputes `binds` and never reads
it; the implementation forty lines away did the opposite. That is the same shape as
the gap in `runtime` the whole proposal is about: a property stated in one place and
contradicted by the thing that actually runs.

The rule now recomputes the binding from REPORT_DATA and accepts either commitment a
producer could plausibly make, the digest string as written or the raw bytes it names.
Neither is present in any capture we own, so the honest result on real hardware is
self-reported, which is what the corpus reports.

`advisory-binds-cannot-raise-a-claim` is the vector that keeps it honest: it declares
the binding, does not have it, and grades self-reported. Every vector now asserts the
model-claim grade alongside the record grade, because 6.1 is a claim about how the two
relate and a corpus checking only the first was not testing it.

9/9, ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKXk6sx5CzAsHmTEJhk7Gy
@imran-siddique
imran-siddique requested a review from a team as a code owner September 3, 2026 21:12
…estly can

CI caught two things this branch owed the repository, and both controls were right.

`test_every_schema_in_the_repository_is_classified` failed because the draft schema
was added without classifying it. It belongs in BOUNDED_SCHEMAS for the same reason
the file it was generated from does: it is `schema/trace-claim.json` plus one member,
`runtime.evidence` adds no integer field, so every integer in it is the bounded one it
inherited. Verified directly rather than assumed: zero unbounded integer nodes.

`test_every_vector_set_on_disk_is_measured_somewhere` failed because a new vector set
appeared with nothing grading it. That would not have fired in CI, because the vectors
were gitignored and the check keys on a directory holding JSON, which makes it a
landmine rather than a non-issue: it fires for anyone who runs the generator. Writing
an RFC about inert fixtures and then shipping an unmeasured set was not an option.

So the set is now real and measured:

  - The signing key is fixed and published, so the corpus is reproducible and
    regeneration is byte-identical. A fresh key per run makes every vector churn and
    makes "regenerate and diff" useless as a review. Verified by regenerating twice.
  - Vectors are committed, wrapped as {expected, record} so the expectation travels
    with the vector. `record` stays a clean TRACE record, because a vector carrying an
    extra top-level member would fail the schema the corpus exists to exercise.
  - tests/test_runtime_evidence_vectors.py asserts schema validity, signature validity,
    evidence shape, and that no vector reaches the top grade, which pins 7.2 so it
    fails deliberately the day a capture binds a signing key rather than going stale.

What that test does NOT do is verify the quotes, and it says so in its docstring
rather than reaching for a skipped test. Quote verification needs a TDX verifier,
TRACE does not ship one, and section 8 argues it should not start; that half runs in
generate.py against agent-manifest's verifier. A test that quietly skipped the
hardware half would look like coverage and be none, which is the defect the whole
proposal is about.

Registered in MEASURED_ELSEWHERE rather than SETS: the adequacy criteria grade a set
on accept/reject outcomes, and this set's outcomes are three assurance grades, so
`trivially_satisfied_by` would be comparing against the wrong two unconditional
implementations.

Local: 39/39 on the new tests, 51/51 with adequacy, ruff clean. The four remaining
failures on this machine are Windows path-separator and line-ending artifacts in
pre-existing tests, unrelated to this branch and green on Linux CI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKXk6sx5CzAsHmTEJhk7Gy

@lywinged lywinged left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested on efad21c. Item 1 fails the suite and item 2 is a rule contradicted by its own reference implementation; the rest are edits to the text. All of it reproduces at the pins below.

Reproduced at the same pins: this head efad21c on base 5c69dc1, agent-manifest at 9348097, cryptography 50.0.1. generate.py prints nine rows matching the table in §7, the accept vector measures 11,923 bytes compact against 1,184 without evidence, and the two captures parse to one MRTD with different REPORT_DATA, so the §7.1 limit reproduces from the captures and not from anything the script does. For the next run: the imported verifier reads not_valid_before_utc at _cert_chain.py line 59, which cryptography 41 does not have, so 41 fails with an AttributeError. agent-manifest's pyproject pins cryptography>=42,<51, but the script imports it by path rather than installing it, and the README's own dependency line names no version.

Four items, each visible from the repository's own checks or text.

  1. The suite fails. tests/test_safe_integer_range.py::test_every_schema_in_the_repository_is_classified enumerates schema/ and requires every file to be listed as bounded or unbounded, and trace-claim-v0.3-draft.json is in neither list. 1 failed, 1,166 passed, 1 skipped.

  2. Rule 2 and the reference implementation disagree. §4 rule 2 says a verifier with no implementation for a format "treats the record as if the block were absent" and grades unattested. §7 says generate.py implements §4, and line 143 raises Reject for any format other than tdx-quote-v4, so a record carrying sev-snp-report-v2 or tpm2-quote, both in the draft enum, is refused rather than downgraded. Either the script grades unattested on an unknown format or rule 2 changes; the rule's second sentence argues for the first.

  3. Line 6 of the RFC says 8 vectors. §7 counts 9/9, the README table has nine rows, and the PR body says nine; the ninth came in with the second commit, which updated §7 and the README but not line 6.

  4. The other two RFCs are in the mkdocs nav, at lines 226 and 227 of mkdocs.yml; this one is not yet. mkdocs build --strict passes with or without the entry.

Two things the rules leave unstated, found by running inputs outside the nine.

REPORT_DATA is 64 bytes, and appraise at line 177 and grade_model_claim at line 208 both compare the first 32 against their candidate. With both comparisons on the same 32 bytes, attested and model claim: attested cannot both hold for one record. binds being a single enum suggests that is intended; if so, §6 should say a model claim can be attested only on a platform-attested record. Both captures carry zeros in the second 32 bytes, so nothing in the corpus reaches the question, and the text does not say whether that half is reserved, ignored, or where a second binding would go.

§5.1's cost is measured on the 8,000-byte capture files, not on the quote structure. Both files declare a structure that ends at byte 4,935, and the remaining 3,065 bytes are zeros the verifier never reads: any prefix of 4,935 bytes or more verifies, bytes appended verify, and a byte flipped inside the tail verifies. Trimmed to the structure, the quote is 6,580 characters and the accept vector is 7,836 bytes compact rather than 11,923. §5.1 can say which of the two it measures.

The same fact bears on the schema's "Byte-identical to what the hardware produced". No verifier can check that against the original, so the description is a producer obligation; the text should say so, or say what a verifier may do with bytes past the declared structure, since the corpus itself carries 3,065 of them. collateral is the same shape: §3 says the record should state it, §4 never reads it, so required on a TDX record grades the same as embedded. Rule 3 already refuses a platform the evidence does not root; the same check would refuse required on tdx-quote-v4.

None of this touches the shape of the proposal. The envelope-first order, the middle grade, and the recompute-never-read rule in §6.1 all do what the text says, and the limit vector reports what the rules do.

Tool-assisted: the runs and this write-up.

Review items 2, 3 and 4 from @lywinged.

Rule 2 and the reference implementation disagreed. Section 4 rule 2 says a
verifier with no implementation for a format treats the record as if the
block were absent, and rule 5 grades that unattested. generate.py raised
Reject for any format other than tdx-quote-v4, so a record carrying
sev-snp-report-v2 or tpm2-quote, both in the draft enum, was refused rather
than downgraded. The by-reference branch six lines below already returned
unattested with exactly that reasoning, so the script disagreed with itself.

The rule is the one that was right: refusing would make a verifier's coverage
gap indistinguishable from bad evidence.

Adds downgrade-unsupported-format so the behaviour is in the corpus rather
than only in the text. It carries the genuine quote bytes under a
sev-snp-report-v2 label with platform amd-sev-snp, which also demonstrates
rule 2 short-circuiting before rule 3: the verifier cannot read the evidence,
so it never reaches the question of whether platform and evidence agree.
Corpus is 10/10. The size pin in test_the_corpus_did_not_shrink moves 9 -> 10,
which is the visible act that test exists to force.

Line 6 said 8 vectors while section 7, the README and the PR body said nine;
now 10 everywhere. Adds the mkdocs nav entry the other two RFCs have.

Signed-off-by: Imran Siddique <imran.siddique@opaque.co>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BiraRPG9NcLDZsNSmSXxE7
@imran-siddique

Copy link
Copy Markdown
Member Author

@lywinged this is the review I wanted and did not get from myself. All four items reproduce, three are fixed in d0fdf88, and the two things you found by running inputs outside the corpus are the more valuable half. Answering everything.

Item 1, already fixed before your review landed

dddcc70 added schema/trace-claim-v0.3-draft.json to BOUNDED_SCHEMAS, so test_every_schema_in_the_repository_is_classified passes. Your reproduction at efad21c was accurate; the commit crossed with it.

Item 2, you were right and the rule wins

The script disagreed with itself, which is the part I should have caught: generate.py refused any format other than tdx-quote-v4, while the by-reference branch six lines below already returned unattested with the same reasoning rule 2 gives. Fixed in the direction you argued for, and for the reason rule 2's second sentence gives: refusing makes a verifier's coverage gap indistinguishable from bad evidence, and a record carrying sound SNP evidence is not defective because this verifier cannot read SNP.

Added downgrade-unsupported-format so it is in the corpus rather than only in the text. It carries the genuine quote bytes under a sev-snp-report-v2 label with platform: amd-sev-snp, which incidentally demonstrates rule 2 short-circuiting before rule 3: the verifier cannot read the evidence, so it never reaches the question of whether platform and evidence agree. Corpus is 10/10.

The size pin in test_the_corpus_did_not_shrink moves 9 to 10. That test exists to make a shrinking corpus a visible act, and growing it should be equally visible, so bumping the constant is the intended cost rather than a workaround.

Items 3 and 4

Line 6 now says 10, matching §7, the README and the corpus. Nav entry added after the A2A profile at mkdocs.yml:228.

REPORT_DATA's second 32 bytes

You are right that nothing in the corpus reaches it and the text does not say what it is. It is reserved, and the profile should say so explicitly rather than leave it to be discovered from the code.

The reason to reserve rather than define: the first 32 bytes carry the binding this profile is about, the cnf key commitment, and that is the only thing rule 6 needs. A second binding in the remaining 32 would be a second commitment made by the same guest-controlled write, so it inherits exactly the trust properties of the first and adds no independent evidence. Defining it now would invite producers to put something load-bearing there before anyone has said what a verifier may conclude from it, which is the mistake §6 exists to prevent.

I will add a sentence to §5.2 saying the second half is reserved, that a verifier MUST NOT derive a grade from it, and that a producer writing into it is making a claim this profile does not evaluate.

§5.1's byte cost, which is the sharper finding

You are right and the number is wrong in the direction that flatters the proposal. The captures are 8,000 bytes, the declared structure ends at 4,935, and the remaining 3,065 are zeros no verifier reads. So §5.1's 11,923 measures padding as if it were cost.

§5.1 should measure the declared structure, so 7,836 bytes compact. That is the honest number and it is the one an implementer sizing a record cares about, since nobody ships the padding deliberately.

Your related point lands harder. The schema says the quote is "byte-identical to what the hardware produced", and no verifier can check that against an original it does not have, so it is a producer obligation stated as if it were a verifiable property. Two things follow, both of which I will write in:

  • the description says it is a producer obligation, not something a verifier establishes;
  • the text says what a verifier may do with bytes past the declared structure. Given the corpus itself carries 3,065 of them and they verify either way, the honest rule is that a verifier evaluates the declared structure and MUST NOT treat trailing bytes as evidence of anything, in either direction.

collateral

Also right. §3 says a record should state it, §4 never reads it, so required and embedded grade identically on a TDX record. Rule 3 already refuses a platform the evidence does not root, and the same check applied to collateral would refuse required on tdx-quote-v4, where the PCK chain travels inside the quote. I will make that rule 3's second clause rather than a new rule.

What is not in this push

The three text changes above, §5.2's reserved sentence, §5.1's measurement basis, and collateral in rule 3. They are wording rather than behaviour and I would rather land them in one edit after you have said whether the reserved-not-defined call is the one you would make. The code and the corpus are consistent as of d0fdf88.

Also noting your environment finding for anyone who runs this next: the imported verifier reads not_valid_before_utc at _cert_chain.py:59, which cryptography 41 lacks, and the README names no version. I will pin it in the README.

@lywinged lywinged left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d0fdf88 checked: 10/10 against agent-manifest 9348097, and regenerating with the published key reproduces all ten committed vectors byte for byte. Suite 1,214 passed, 1 skipped. Line 6 and mkdocs.yml line 228 as described.

Reserved is the call I would make, for two reasons beyond the one given. The producer already in use writes it that way: TDXProvider.extend_manifest_hash requests sha256(pre_image) || 0x00*32, and the verifier on the other side compares the first 32 bytes only. And rule 6 has to recompute from the record alone, and the record-signing key is the only bound object a record carries, so a second slot would have nothing to hold that a verifier could check offline. On wording: say the verifier does not read the second 32 bytes, rather than requiring them to be zero, so a producer that fills all 64 is not refused for it.

One question the reserved sentence brings forward, for the text and not for this push. §6.1 grounds an attested claim in what hardware measured. grade_model_claim recomputes from REPORT_DATA, which the guest writes when it asks for the quote, so a weights digest committed there is the producer's statement, made earlier and inside the envelope. The registers that hold measurements are the RTMRs. The parser already exposes them, rule 4 reads only the MRTD, and both captures carry RTMR0 to RTMR2 set with RTMR3 zero. Two ways to close it: point the model claim's recompute at a runtime register, or have §6.1 say that a REPORT_DATA commitment is a claim rather than a measurement and grade it so. Either way REPORT_DATA then carries the key and nothing else, and the exclusivity I raised stops being a question.

A smaller one in the same area. Rule 6 speaks of the platform's guest-controlled field. agent-manifest's own notes say that on Azure confidential VMs the guest does not control it, the paravisor binds the vTPM AK there, and the manifest hash is bound by extending a PCR under an AK-signed quote. tpm2-quote is in the enum with no binding rule, so on that path the top grade is unreachable by construction rather than by capture. A line saying the binding rule is per format, as the measurement rule already is, would cover it.

The reserved sentence as drafted in the comment carries an uppercase keyword; line 9 of the RFC says lowercase throughout, and the docs guard wants a citation on the same line as any uppercase one.

I will approve on the head that carries the five text changes: §5.2 reserved, §5.1 measured on the declared structure, the quote description, collateral in rule 3, and the README pin. The two questions above are not conditions.

Tool-assisted: the runs and this write-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants