Skip to content

feat(execution): durable session-independent execution correlation (#565) - #606

Open
Yatsuiii wants to merge 5 commits into
agentrust-io:mainfrom
Yatsuiii:feat/565-execution-state
Open

feat(execution): durable session-independent execution correlation (#565)#606
Yatsuiii wants to merge 5 commits into
agentrust-io:mainfrom
Yatsuiii:feat/565-execution-state

Conversation

@Yatsuiii

@Yatsuiii Yatsuiii commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What

First implementation slice for session-independent execution correlation (#565),
following the design merged in #574. Adds cmcp_runtime.execution: a durable
ExecutionRegistry that atomically reserves (agent_identity, execution_id, action_binding) before upstream invocation, classifies replays and collisions
without re-invoking, and fails closed on restart. Plus the minimal wiring to
consult it on the tool call path and a typed AuditEntry.execution_id field.

Why

Closes part of #565 (correlation and durable state). Per #574, session_id is
the only validated correlation key today and it carries two jobs that stop
fitting together across independent requests. execution_id is the caller-
asserted key that survives independent requests; this slice makes it durable and
gives it fail-closed collision/replay semantics so a fault after upstream
invocation cannot become permission to repeat an irreversible effect.

Design notes for reviewers

  • State machine. admit() (under BEGIN IMMEDIATE) reserves a fresh
    in_flight row or classifies an existing key: replay_in_flight,
    replay_terminal, replay_outcome_unknown, collision_changed_binding. It
    never rewrites a row. finalize() moves in_flight to completed (transport
    delivered) or outcome_unknown (earlier); both terminal and non-replayable.
    recover() seals every still-in_flight row as outcome_unknown at startup.
  • Action binding is deferred to Define canonical bytes for execution action/intent bindings #588. admit() takes the binding as an
    opaque digest string and never computes it; the proxy gets it from an injected
    action_binding_fn. provisional_action_binding ships as an explicit stub
    with an undecided preimage (docstring cites the open question and altrudev's
    proposed envelope) that reuses the existing cmcp_runtime.catalog.approval
    canonicalizer. No second serializer.
  • Requirement 7 is partially met. The Define session-independent execution correlation for post-session MCP #565 thread targeted one transaction
    boundary for the terminal state and its audit evidence, noting AuditStore
    would need a method that writes both under one BEGIN. This slice does not
    do that: the execution row and the audit chain entry are in separate SQLite
    databases. Instead the proxy writes the audit entry first, then finalize().
    The narrower docs: propose session-independent execution correlation (#565) #574 invariant still holds - a terminal transition is durable
    before a later request is classified as replay, because finalize() commits
    its own transaction before any later admit() runs. A crash in the gap, or a
    failure inside finalize(), leaves the row in_flight for recover() to
    seal. The only residual ambiguity is an audit entry showing a definite outcome
    while the registry says outcome_unknown, which is the more conservative
    reading. A real single transaction needs a shared single-writer datastore and
    is out of this slice (disclosed in the spec under "Known limitations").
  • execution_id is bound-checked at ingress (1 to 200 printable non-space ASCII)
    before it can reach the durable key; a violation is denied
    (execution_invalid_execution_id) with no reservation and is not written to
    the audit entry.
  • Known review question: _classify_existing compares bindings with != rather
    than hmac.compare_digest.

Scope held out

HTTP-native transport, Tasks, delegated identity, discovery, broad SDK support.
No claim of exactly-once external execution. #588's binding construction.

Security impact

Touches audit chain integrity and adds replay/collision enforcement.

  • New typed AuditEntry.execution_id field: always serialized, null when
    absent, never synthesized. It is included in the hashed entry body, so entry
    hashes for chains written after this change differ from before; existing
    persisted chains still verify against their own stored payloads.
  • Replay and collision policy lives entirely in ExecutionRegistry; the proxy
    has one call site each for admission and finalization and does not branch on
    policy. A replay or collision is refused before upstream invocation and the
    refusal is audited under the asserted execution_id.
  • Fail-closed on ambiguity: any execution left in_flight by a crash or a
    persistence failure becomes outcome_unknown at the next startup and can never
    admit another invocation. There is no replay window and no expiry.
  • Not addressed here: a single durable transaction spanning the terminal
    execution row and the terminal audit entry (see "Requirement 7" above);
    cross-process reservation fencing beyond SQLite BEGIN IMMEDIATE +
    busy_timeout.

Test plan

  • pytest passes - tests/unit + tests/conformance: 1557 passed, 6
    skipped. The 6 tests/unit/test_startup.py failures are pre-existing TPM
    /dev/tpmrm0 device-permission failures that reproduce on clean main;
    untouched here.
  • ruff check passes (ruff check src tests)
  • mypy passes (mypy src, 69 files)
  • New coverage: tests/unit/test_execution_correlation.py (20, registry
    level incl. an 8-thread reservation race, restart recovery, injected
    persistence failure) and tests/unit/test_execution_correlation_call_path.py
    (12, proxy level: reserve before upstream, no re-invoke on replay/collision,
    audit carries execution_id, finalize on terminal and on upstream fault).

DCO sign-off

  • I certify that I wrote or have the right to submit this contribution, and I
    agree to the Developer Certificate of Origin (https://developercertificate.org).
    All four commits carry Signed-off-by.

🤖 Generated with Claude Code

Yatsuiii and others added 4 commits September 3, 2026 13:11
…st-io#565)

First implementation slice for session-independent execution correlation,
following the design merged in agentrust-io#574 (docs/spec/execution-correlation.md).

ExecutionRegistry is the authoritative execution-lifecycle record: a
SQLite-backed deep module, one instance per process, keyed by
(agent_identity, execution_id) with a PRIMARY KEY on the pair.

- admit() runs under BEGIN IMMEDIATE and either reserves a fresh in_flight
  row or classifies an existing key as a replay (in_flight / terminal /
  outcome_unknown) or a collision (changed action binding). It never
  rewrites a row and never returns "proceed" for one.
- finalize() moves in_flight to completed (transport delivered a response)
  or outcome_unknown (anything earlier). Both terminal, neither replayable.
  A second call is a no-op.
- recover() runs once at startup and seals every still-in_flight row as
  outcome_unknown, so a crash between admit and finalize can never admit
  another invocation.

The action binding reaches admit() as an opaque digest string; the registry
only stores and byte-compares it. Its canonical construction is issue agentrust-io#588's.
execution/binding.py carries provisional_action_binding, an explicit stub
with an undecided preimage that reuses the existing RFC 8785/JCS
canonicalizer (cmcp_runtime.catalog.approval) so no second serializer is
introduced. valid_execution_id bounds the identifier to 1-200 printable
non-space ASCII characters before it can reach the durable key.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…#565)

execution_id is a typed AuditEntry field, always serialized (null when the
caller asserted none), never a detail key. A present value came through the
validated admission path in ExecutionRegistry; null means no assertion was
made and none was synthesized. AuditChain.append gains the matching keyword.

Per the agentrust-io#574 design: the TRACE Claim does not enumerate execution_id values;
the audit entry carries the join key so bundles stay joinable offline.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…gentrust-io#565)

Wire ExecutionRegistry into the tool call path.

- server.py reads _cmcp.execution_id beside workflow_id; the two stay
  independent. A non-string is treated as absent.
- proxy._admit_execution reserves the execution identity at step 3a.5,
  immediately before upstream invocation. execution_id is bound-checked
  first (valid_execution_id); a malformed value is denied
  (execution_invalid_execution_id) with no reservation and is not written
  to the audit entry. The action binding comes from an injected
  action_binding_fn, so this module never encodes the agentrust-io#588 preimage.
- A replay or collision is refused here and never reaches upstream. The
  refusal is audited under the asserted execution_id.
- finalize() is called from the single terminal audit write
  (_append_call_terminal), so replay and collision policy is not spread
  across handlers. completed only when the transport delivered a response;
  anything earlier is outcome_unknown.
- cli.build_server constructs one process-wide registry, runs recover()
  before the gateway serves traffic, and injects provisional_action_binding.

A terminal transition is durable before a later request is classified as
replay, because finalize() commits its own transaction before any later
admit() runs. The terminal audit entry and the execution row are in
separate SQLite databases and do not share one transaction; a crash in the
gap leaves the row in_flight for recover() to seal. See
docs/spec/execution-correlation.md "Known limitations".

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ntation slice

Add an "Implementation (first slice)" section to
docs/spec/execution-correlation.md describing what this slice builds:
ExecutionRegistry as the authoritative state owner, the admit / finalize /
recover contract, the opaque-digest binding seam, and execution_id ingress
validation.

The "Known limitations" subsection records that the terminal audit entry and
the execution row are in separate SQLite databases with no shared
transaction, that there is no cross-process reservation fencing beyond
SQLite BEGIN IMMEDIATE plus busy_timeout, and that this slice reports a
shared asserted execution_id without claiming exactly-once external
execution.

The action-binding construction (preimage, JCS member ordering, digest
representation) stays deferred to agentrust-io#588; this doc only describes what the
slice consumes.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Yatsuiii
Yatsuiii requested review from a team as code owners September 3, 2026 07:49
@codecov-commenter

codecov-commenter commented Sep 3, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Yatsuiii
Yatsuiii force-pushed the feat/565-execution-state branch from 89bf09b to 71104af Compare September 3, 2026 09:15
…e rollback paths

Codecov flagged nine uncovered lines in ExecutionRegistry: the recover()
rollback path and an admit() sqlite3.IntegrityError handler for a lost insert
race.

The IntegrityError handler was unreachable. Verified against two independent
connections on one database file: the second BEGIN IMMEDIATE blocks on the
first connection's RESERVED lock, and once the first commits, the second's
SELECT runs inside its own transaction and observes the committed row, so it
classifies rather than reaching the INSERT. When the lock is held past
busy_timeout the caller gets sqlite3.OperationalError, not IntegrityError.
Within one process the registry's own threading.Lock serialises. No path
produces the exception the handler caught, so it is removed rather than tested.

The spec said "a lost insert race re-reads and classifies", which described a
path that cannot occur. It now says a competing writer either observes the
committed reservation or fails closed if the lock timeout is exceeded.

Tests added for the paths that are reachable:

- admission rollback under an injected persistence failure, asserting no row
  is left behind and that a later admit still succeeds;
- recovery rollback under an injected persistence failure, asserting the row
  stays in_flight for a later recover() rather than half-sealed;
- two independent ExecutionRegistry instances on one file admitting the same
  key simultaneously, asserting exactly one ADMITTED and one REPLAY_IN_FLIGHT.

Execution package coverage 92% to 100%.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants