docs(rfc): propose runtime evidence, and what a verifier may conclude without it - #277
docs(rfc): propose runtime evidence, and what a verifier may conclude without it#277imran-siddique wants to merge 4 commits into
Conversation
… without it
A Trust Record states a hardware platform and a measurement, and carries nothing a
relying party can check either against. `runtime` is additionalProperties:false over
five members, none of which is the quote, the certificate chain, or the report
signature. `verify_record()` performs no attestation verification, and the package
contains none: "quote" appears in src/agentrust_trace/ only inside comments. That is
not an implementation gap, because there is nothing in the record for such code to
consume.
The specification already reasons about this, in one direction only. 3.1.1 forces
software-only on an assembled record because "an importer holding someone else's log
has no quote to present". 3.1.2 rule 3 forbids treating a resolved reference as
attested evidence, naming assurance laundering as the reason. Both govern pointers to
things outside the record. Neither governs `runtime`, where the same move is not just
possible but is all that is on offer: rim_uri is a pointer, measurement is a
transcription, appraisal.verifier names someone else's verdict. This proposal is the
project's own doctrine applied to the block it was never applied to.
It is also a requirement already made and never made checkable. docs/trust-levels.md
line 50 requires a Level 1 signing key to be generated inside a verified TEE, and
spec 5 maps `runtime` to "RATS Evidence + vendor RIM". Nothing represents either. The
requirement, the mapping and the schema disagree, and the schema is what runs.
Adds a `runtime.evidence` member (draft schema generated from the v0.2 one, so the diff
is one member deep), seven verification rules, and three grades: unattested,
platform-attested, attested. The middle grade was forced by the artifacts rather than
chosen, and 6.1 keeps a grade on the record from propagating to the claims inside it,
which is the same anti-laundering rule turned inward.
The corpus is eight vectors over two GENUINE Intel TDX v4 quotes from a GCP C3 guest,
verified by agent-manifest's verifier imported unmodified. TRACE does not implement
attestation and this does not start.
Two findings, both kept rather than smoothed:
- The substitution vector was written to be a rejection and is not one. Both captures
come from one TD, so they share an MRTD, and the measurement rule cannot separate
two quotes that agree on the measurement. The rule binds a record to a measurement
and never to a quote. That caps the middle grade at "genuine silicon reporting this
measurement" and is the argument for the top grade existing. A synthetic corpus
would have minted different measurements and passed.
- The top grade is currently unreachable. No capture we hold binds a record-signing
key, and the pre-image behind the bindings we do have was never committed, so
REPORT_DATA is a value that verifiably came from the TEE and cannot be opened. Both
are gaps in the capture procedure, cheap to close on the next run, and cheaper to
say now than to discover during adoption.
Informative throughout, lowercase keywords, per CONTRIBUTING.md. Binds nothing. No
existing field, schema or record changes; a record without `evidence` grades
unattested, which is a name for what every v0.2 record already was.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKXk6sx5CzAsHmTEJhk7Gy
Two defects, one of them the kind this proposal is about. The schema enum is `weights-digest` and `grade_model_claim` compared against `weights_digest`, so the branch could not fire. No vector set `binds`, so nothing noticed. Correcting the spelling would have made it worse. Reading `binds` at all lets a producer raise its own model claim by writing a string into a record, which is the assurance laundering section 6.1 exists to forbid, reintroduced inside the section that forbids it. Section 3 already said a verifier recomputes `binds` and never reads it; the implementation forty lines away did the opposite. That is the same shape as the gap in `runtime` the whole proposal is about: a property stated in one place and contradicted by the thing that actually runs. The rule now recomputes the binding from REPORT_DATA and accepts either commitment a producer could plausibly make, the digest string as written or the raw bytes it names. Neither is present in any capture we own, so the honest result on real hardware is self-reported, which is what the corpus reports. `advisory-binds-cannot-raise-a-claim` is the vector that keeps it honest: it declares the binding, does not have it, and grades self-reported. Every vector now asserts the model-claim grade alongside the record grade, because 6.1 is a claim about how the two relate and a corpus checking only the first was not testing it. 9/9, ruff clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LKXk6sx5CzAsHmTEJhk7Gy
…estly can
CI caught two things this branch owed the repository, and both controls were right.
`test_every_schema_in_the_repository_is_classified` failed because the draft schema
was added without classifying it. It belongs in BOUNDED_SCHEMAS for the same reason
the file it was generated from does: it is `schema/trace-claim.json` plus one member,
`runtime.evidence` adds no integer field, so every integer in it is the bounded one it
inherited. Verified directly rather than assumed: zero unbounded integer nodes.
`test_every_vector_set_on_disk_is_measured_somewhere` failed because a new vector set
appeared with nothing grading it. That would not have fired in CI, because the vectors
were gitignored and the check keys on a directory holding JSON, which makes it a
landmine rather than a non-issue: it fires for anyone who runs the generator. Writing
an RFC about inert fixtures and then shipping an unmeasured set was not an option.
So the set is now real and measured:
- The signing key is fixed and published, so the corpus is reproducible and
regeneration is byte-identical. A fresh key per run makes every vector churn and
makes "regenerate and diff" useless as a review. Verified by regenerating twice.
- Vectors are committed, wrapped as {expected, record} so the expectation travels
with the vector. `record` stays a clean TRACE record, because a vector carrying an
extra top-level member would fail the schema the corpus exists to exercise.
- tests/test_runtime_evidence_vectors.py asserts schema validity, signature validity,
evidence shape, and that no vector reaches the top grade, which pins 7.2 so it
fails deliberately the day a capture binds a signing key rather than going stale.
What that test does NOT do is verify the quotes, and it says so in its docstring
rather than reaching for a skipped test. Quote verification needs a TDX verifier,
TRACE does not ship one, and section 8 argues it should not start; that half runs in
generate.py against agent-manifest's verifier. A test that quietly skipped the
hardware half would look like coverage and be none, which is the defect the whole
proposal is about.
Registered in MEASURED_ELSEWHERE rather than SETS: the adequacy criteria grade a set
on accept/reject outcomes, and this set's outcomes are three assurance grades, so
`trivially_satisfied_by` would be comparing against the wrong two unconditional
implementations.
Local: 39/39 on the new tests, 51/51 with adequacy, ruff clean. The four remaining
failures on this machine are Windows path-separator and line-ending artifacts in
pre-existing tests, unrelated to this branch and green on Linux CI.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKXk6sx5CzAsHmTEJhk7Gy
lywinged
left a comment
There was a problem hiding this comment.
Changes requested on efad21c. Item 1 fails the suite and item 2 is a rule contradicted by its own reference implementation; the rest are edits to the text. All of it reproduces at the pins below.
Reproduced at the same pins: this head efad21c on base 5c69dc1, agent-manifest at 9348097, cryptography 50.0.1. generate.py prints nine rows matching the table in §7, the accept vector measures 11,923 bytes compact against 1,184 without evidence, and the two captures parse to one MRTD with different REPORT_DATA, so the §7.1 limit reproduces from the captures and not from anything the script does. For the next run: the imported verifier reads not_valid_before_utc at _cert_chain.py line 59, which cryptography 41 does not have, so 41 fails with an AttributeError. agent-manifest's pyproject pins cryptography>=42,<51, but the script imports it by path rather than installing it, and the README's own dependency line names no version.
Four items, each visible from the repository's own checks or text.
-
The suite fails.
tests/test_safe_integer_range.py::test_every_schema_in_the_repository_is_classifiedenumeratesschema/and requires every file to be listed as bounded or unbounded, andtrace-claim-v0.3-draft.jsonis in neither list. 1 failed, 1,166 passed, 1 skipped. -
Rule 2 and the reference implementation disagree. §4 rule 2 says a verifier with no implementation for a
format"treats the record as if the block were absent" and gradesunattested. §7 saysgenerate.pyimplements §4, and line 143 raisesRejectfor any format other thantdx-quote-v4, so a record carryingsev-snp-report-v2ortpm2-quote, both in the draft enum, is refused rather than downgraded. Either the script gradesunattestedon an unknown format or rule 2 changes; the rule's second sentence argues for the first. -
Line 6 of the RFC says 8 vectors. §7 counts 9/9, the README table has nine rows, and the PR body says nine; the ninth came in with the second commit, which updated §7 and the README but not line 6.
-
The other two RFCs are in the mkdocs nav, at lines 226 and 227 of
mkdocs.yml; this one is not yet.mkdocs build --strictpasses with or without the entry.
Two things the rules leave unstated, found by running inputs outside the nine.
REPORT_DATA is 64 bytes, and appraise at line 177 and grade_model_claim at line 208 both compare the first 32 against their candidate. With both comparisons on the same 32 bytes, attested and model claim: attested cannot both hold for one record. binds being a single enum suggests that is intended; if so, §6 should say a model claim can be attested only on a platform-attested record. Both captures carry zeros in the second 32 bytes, so nothing in the corpus reaches the question, and the text does not say whether that half is reserved, ignored, or where a second binding would go.
§5.1's cost is measured on the 8,000-byte capture files, not on the quote structure. Both files declare a structure that ends at byte 4,935, and the remaining 3,065 bytes are zeros the verifier never reads: any prefix of 4,935 bytes or more verifies, bytes appended verify, and a byte flipped inside the tail verifies. Trimmed to the structure, the quote is 6,580 characters and the accept vector is 7,836 bytes compact rather than 11,923. §5.1 can say which of the two it measures.
The same fact bears on the schema's "Byte-identical to what the hardware produced". No verifier can check that against the original, so the description is a producer obligation; the text should say so, or say what a verifier may do with bytes past the declared structure, since the corpus itself carries 3,065 of them. collateral is the same shape: §3 says the record should state it, §4 never reads it, so required on a TDX record grades the same as embedded. Rule 3 already refuses a platform the evidence does not root; the same check would refuse required on tdx-quote-v4.
None of this touches the shape of the proposal. The envelope-first order, the middle grade, and the recompute-never-read rule in §6.1 all do what the text says, and the limit vector reports what the rules do.
Tool-assisted: the runs and this write-up.
Review items 2, 3 and 4 from @lywinged. Rule 2 and the reference implementation disagreed. Section 4 rule 2 says a verifier with no implementation for a format treats the record as if the block were absent, and rule 5 grades that unattested. generate.py raised Reject for any format other than tdx-quote-v4, so a record carrying sev-snp-report-v2 or tpm2-quote, both in the draft enum, was refused rather than downgraded. The by-reference branch six lines below already returned unattested with exactly that reasoning, so the script disagreed with itself. The rule is the one that was right: refusing would make a verifier's coverage gap indistinguishable from bad evidence. Adds downgrade-unsupported-format so the behaviour is in the corpus rather than only in the text. It carries the genuine quote bytes under a sev-snp-report-v2 label with platform amd-sev-snp, which also demonstrates rule 2 short-circuiting before rule 3: the verifier cannot read the evidence, so it never reaches the question of whether platform and evidence agree. Corpus is 10/10. The size pin in test_the_corpus_did_not_shrink moves 9 -> 10, which is the visible act that test exists to force. Line 6 said 8 vectors while section 7, the README and the PR body said nine; now 10 everywhere. Adds the mkdocs nav entry the other two RFCs have. Signed-off-by: Imran Siddique <imran.siddique@opaque.co> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BiraRPG9NcLDZsNSmSXxE7
|
@lywinged this is the review I wanted and did not get from myself. All four items reproduce, three are fixed in Item 1, already fixed before your review landed
Item 2, you were right and the rule winsThe script disagreed with itself, which is the part I should have caught: Added The size pin in Items 3 and 4Line 6 now says 10, matching §7, the README and the corpus. Nav entry added after the A2A profile at REPORT_DATA's second 32 bytesYou are right that nothing in the corpus reaches it and the text does not say what it is. It is reserved, and the profile should say so explicitly rather than leave it to be discovered from the code. The reason to reserve rather than define: the first 32 bytes carry the binding this profile is about, the I will add a sentence to §5.2 saying the second half is reserved, that a verifier MUST NOT derive a grade from it, and that a producer writing into it is making a claim this profile does not evaluate. §5.1's byte cost, which is the sharper findingYou are right and the number is wrong in the direction that flatters the proposal. The captures are 8,000 bytes, the declared structure ends at 4,935, and the remaining 3,065 are zeros no verifier reads. So §5.1's 11,923 measures padding as if it were cost. §5.1 should measure the declared structure, so 7,836 bytes compact. That is the honest number and it is the one an implementer sizing a record cares about, since nobody ships the padding deliberately. Your related point lands harder. The schema says the quote is "byte-identical to what the hardware produced", and no verifier can check that against an original it does not have, so it is a producer obligation stated as if it were a verifiable property. Two things follow, both of which I will write in:
|
lywinged
left a comment
There was a problem hiding this comment.
d0fdf88 checked: 10/10 against agent-manifest 9348097, and regenerating with the published key reproduces all ten committed vectors byte for byte. Suite 1,214 passed, 1 skipped. Line 6 and mkdocs.yml line 228 as described.
Reserved is the call I would make, for two reasons beyond the one given. The producer already in use writes it that way: TDXProvider.extend_manifest_hash requests sha256(pre_image) || 0x00*32, and the verifier on the other side compares the first 32 bytes only. And rule 6 has to recompute from the record alone, and the record-signing key is the only bound object a record carries, so a second slot would have nothing to hold that a verifier could check offline. On wording: say the verifier does not read the second 32 bytes, rather than requiring them to be zero, so a producer that fills all 64 is not refused for it.
One question the reserved sentence brings forward, for the text and not for this push. §6.1 grounds an attested claim in what hardware measured. grade_model_claim recomputes from REPORT_DATA, which the guest writes when it asks for the quote, so a weights digest committed there is the producer's statement, made earlier and inside the envelope. The registers that hold measurements are the RTMRs. The parser already exposes them, rule 4 reads only the MRTD, and both captures carry RTMR0 to RTMR2 set with RTMR3 zero. Two ways to close it: point the model claim's recompute at a runtime register, or have §6.1 say that a REPORT_DATA commitment is a claim rather than a measurement and grade it so. Either way REPORT_DATA then carries the key and nothing else, and the exclusivity I raised stops being a question.
A smaller one in the same area. Rule 6 speaks of the platform's guest-controlled field. agent-manifest's own notes say that on Azure confidential VMs the guest does not control it, the paravisor binds the vTPM AK there, and the manifest hash is bound by extending a PCR under an AK-signed quote. tpm2-quote is in the enum with no binding rule, so on that path the top grade is unreachable by construction rather than by capture. A line saying the binding rule is per format, as the measurement rule already is, would cover it.
The reserved sentence as drafted in the comment carries an uppercase keyword; line 9 of the RFC says lowercase throughout, and the docs guard wants a citation on the same line as any uppercase one.
I will approve on the head that carries the five text changes: §5.2 reserved, §5.1 measured on the declared structure, the quote description, collateral in rule 3, and the README pin. The two questions above are not conditions.
Tool-assisted: the runs and this write-up.
The gap
A Trust Record states a hardware platform and a measurement, and carries nothing a relying party can check either against.
runtimeisadditionalProperties: falseover five members, none of which is the quote, the certificate chain, or the report signature.appraisalis a status enum plus averifierURI, which is a pointer to a party who says they checked.verify_record()performs no attestation verification, and the package contains none: the stringquoteappears insrc/agentrust_trace/only inside comments. That is not an implementation gap. There is nothing in the record for such code to consume.So the chain a relying party actually gets is: the record is signed, the signature is sound, and it attests authorship. Every hardware claim inside is the issuer's word, and a verifier reporting "TEE-attested" on that basis is repeating the issuer's assertion in the verifier's voice.
Why this is our own doctrine, not a new idea
The spec already reasons about assurance laundering, carefully, and in one direction only.
§3.1.1 forces
software-onlyonto an assembled record because "an importer holding someone else's log has no quote to present". §3.1.2 rule 3 forbids treating a resolvedreferencesentry as attested evidence, naming assurance laundering as the reason. Both govern pointers to things outside the record. Neither governsruntime, where the same move is not merely possible but is all that is on offer:rim_uriis a pointer,measurementis a transcription,appraisal.verifiernames someone else's verdict.It is also a requirement already made and never made checkable.
docs/trust-levels.md:50requires a Level 1 signing key to be generated inside a verified TEE, andspec/trace-v0.2.md§5 mapsruntimeto "RATS Evidence + vendor RIM". Nothing represents either. The requirement, the mapping, and the schema disagree, and the schema is what runs.What this adds
runtime.evidencecarrying the quote, inschema/trace-claim-v0.3-draft.json, generated from the v0.2 schema so the diff is exactly one member deep.unattested,platform-attested,attested. The middle one was forced by the artifacts, not chosen.Additive. No existing field, schema, or record changes. A record without
evidencegradesunattested, which is a name for what every v0.2 record already was.Conformance material
Nine vectors over two genuine Intel TDX v4 quotes from a GCP C3 guest, verified by
agent-manifest's verifier imported unmodified. Nothing is minted. TRACE does not implement attestation and this proposal does not start.Three findings, all kept rather than smoothed
The substitution vector was written to be a rejection and is not one. Both captures come from one TD, so they share an MRTD, and the measurement rule cannot separate two quotes that agree on the measurement. The rule binds a record to a measurement, never to a quote. That caps the middle grade at "genuine silicon reporting this measurement" and is the argument for the top grade existing. A synthetic corpus would have minted different measurements and passed.
The top grade is currently unreachable. No capture we hold binds a record-signing key, and the pre-image behind the bindings we do have was never committed, so
REPORT_DATAis a value that verifiably came from the TEE and cannot be opened. Both are capture-procedure gaps, cheap to close on the next TDX run, and cheaper to say now than to discover during adoption.The first draft of §6.1 contained the bug §6.1 forbids. It graded the model claim by reading
evidence.binds, which would let a producer raise its own claim by writing a string. §3 already said a verifier recomputesbindsand never reads it; the implementation forty lines away did the opposite. Same shape as the gap inruntimethis whole proposal is about. Fixed in the second commit, with a vector that declares a binding it does not have and stays self-reported.Status
Informative, lowercase keywords, per
CONTRIBUTING.md. Binds nothing. It does not resolve §5.2: the reference producer binds a manifest digest rather than the signing key today, and a profile is not adopted by writing down which side should move.🤖 Generated with Claude Code
https://claude.ai/code/session_01LKXk6sx5CzAsHmTEJhk7Gy