Skip to content

Make approval provenance auditable - #39

Merged
raghubetina merged 1 commit into
mainfrom
codex/fix-action-provenance-0.1.2
Aug 18, 2026
Merged

Make approval provenance auditable#39
raghubetina merged 1 commit into
mainfrom
codex/fix-action-provenance-0.1.2

Conversation

@raghubetina

Copy link
Copy Markdown
Contributor

Summary

  • require pre-Compile read-backs to report every attempted tool or command and its outcome
  • require filesystem observations to be backed by successful tool results
  • keep local inspection provenance separate from First Draft, Compile, and Publication effects
  • refresh the current 0.1.2 package digest without changing historical evidence

Verification

  • sh script/check: 70/70
  • exact CLI package check: pass
  • deterministic pack SHA-256: ef59d49a20e3704adfcfd3b7e9a345b2f3b1c4ebe4b106315932dba7400b9121
  • ordinary and adversarial documentation reviews: pass

Require pre-Compile read-backs to distinguish attempted, failed, denied, and successful actions. This prevents a semantic approval from contradicting its retained tool ledger while keeping First Draft, Compile, and Publication effects separate.
@raghubetina
raghubetina merged commit 62998ba into main Aug 18, 2026
2 checks passed
@raghubetina
raghubetina deleted the codex/fix-action-provenance-0.1.2 branch August 18, 2026 11:43
@raghubetina

Copy link
Copy Markdown
Contributor Author

Post-merge review, hosted CI green. This PR required pre-Compile read-backs to report every attempted tool or
command with its outcome, and required filesystem observations to be backed by successful tool results. Four
days later #40 removed the exhaustive ledger, the shell classification, and the environmental-inactivity
proofs, and #41 trimmed further, so most of this PR's added ceremony is now historical. Review it as a
worthwhile experiment that found the ceiling.

What survived is the good part: the principle that a claimed observation must trace to a retained successful
tool result (an agent saying "the Plan has three Entities" must have actually read the file, and the evidence
must show the read). That grounding requirement persists in the current smoke through the retained transcript
and the read-back-before-approval structure. What did not survive is the exhaustive enumeration, which made
honest runs fail on bookkeeping and produced evidence whose volume outran its value.

No action needed; noting the supersession so a reader of this diff does not implement against it.

@raghubetina

Copy link
Copy Markdown
Contributor Author

This PR added heavyweight auditability to an agent approval flow, and most of it was rolled back within a
week. That makes it a better lesson than a success would be.

The intent was sound. When an agent claims "I inspected the staged Plan and here is its semantic model," you
want provenance: which files it read, with what tool, and whether the read succeeded. Ungrounded claims are
how agents hallucinate their way through checklists. So the PR required the agent to account for every
attempted action and required every filesystem observation to match a retained successful result.

Why it did not hold: completeness requirements scale badly. "Report every attempted tool or command" turns
each incidental ls into a compliance obligation, and any mismatch between the agent's account and the ledger
fails the run, including mismatches that are recording artifacts rather than deception. The audit surface grew
until the qualification failed on paperwork while the substantive behavior (read the Plan, present the model,
wait for approval) was fine. The follow-up PRs kept the grounding principle, observations must trace to
retained evidence, and dropped the exhaustiveness.

The transferable rule: audit claims, not activity. Demand evidence for each assertion the decision actually
depends on, and resist the urge to log everything as a proxy for trust. In Rails terms, you validate the
attributes that matter for the save; you do not diff the whole object graph on every write.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant