test(sandbox): prove #455 end to end against a live Daytona sandbox

[khaliqgant]

Verification lane for #457. I did not write that PR and this does not change it — this is the missing evidence that it works on the real path.

Verdict: CONFIRMED. Based on fix/455-state-json-single-writer so the diff here is only the proof.

Why this exists

#457's evidence is sandboxInitialSyncGuard, a Go transcription of the sandbox readiness guard. The transcription is faithful — but no sandbox had ever provisioned with this build, so the claim was untested where it actually fails. This closes that gap.

The guard here is imported from @agent-relay/sandbox at origin/main cfdf801 (v0.1.14, the version cloud/packages/core pins), not reimplemented, and invoked through buildRelayfileMountInitialSyncBackgroundShell — the detached launcher, because Daytona's exec proxy read-times-out around 120s and cannot host a real initial sync in the foreground.

Both exit codes

Sandbox 8a7d6049-ca63-447b-abdf-f750f121c994, remote root /github/repos/AgentWorkforce/relay, 5,685 files. Both arms run the identical generated command against the identical subtree and differ only in which relayfile-mount is first on PATH.

	arm A — snapshot-baked	arm B — built from 4e3c1105
guard exit	75	0
files mirrored	2,000	5,685
bootstrap	non-null, filesSynced: 2000	key absent
lastSuccessfulReconcileAt	2026-09-02T12:18:24.303486294Z	2026-09-02T12:47:48.742036968Z
status	bootstrapping	ready
state.json sha256	3d59ae71dff2…37dc1d43	39329c7c14f5…14dad359

Raw .relay/state.json bytes for both arms are committed under docs/evidence/daytona-455-initial-sync-readiness-20260902/raw/, with the sha256 taken inside the sandbox.

All four claimed steps in one run (arm A):

bootstrap file budget reached (2000 files this cycle, max 2000); yielding at entry 170
mount full-tree traversal summary ... files_seen=2000 traversal_complete=false traversal_failed=false
mount sync cycle completed                                  <-- --once reports SUCCESS
relayfile initial sync paused before complete readiness      <-- guard exits 75

And the fix (arm B): 2,000 → 4,000 → 5,685 across three resume cycles, then initial sync: bootstrap complete.

The control is tight. Both arms' first cycles are numerically identical — list_calls=6 entries_seen=3986 files_seen=2000 directories_seen=1986 bytes_seen=26734874, both yielding at entry 170 with 887 directories pending. Same fixture, same budget, same stopping point; they diverge only in what happens next.

The harness fails closed

scripts/test-sandbox-initial-sync-readiness-e2e.mjs exits 2 (UNKNOWN), not 0, when it cannot provision, when an arm times out, or when arm A passes — a must-fail control that does not fail means the fixture never crossed the 2,000-file budget and neither arm proves anything.

Three things found on the way that are not in the claim

1. Publishing fix(mount): make --once reach a complete reconcile, and stop the two state.json writers clobbering each other #457 will not reach any sandbox. The mount binary is baked into the Daytona snapshot, not resolved at provision time — snapshot names encode it, and the newest snapshot built 2026-09-02 still carries v0.10.50. A snapshot rebuild and a fleet pin bump are required on top of the merge.

2. The snapshot's label disagrees with its contents. Named v0.10.50, but @relayfile/mount-linux-x64 inside is 0.10.51, and three different relayfile-mount binaries with three different sha256s are present. Which one runs depends on PATH.

3. tokenIngress: 'env' in @agent-relay/sandbox is broken. It renders RELAYFILE_MOUNT_TOKEN=, but cmd/relayfile-mount/main.go reads only RELAYFILE_TOKEN / RELAYFILE_MOUNT_CREDS_FILE. No build — including this PR's head — reads it. Observed directly: token is required (--token, RELAYFILE_TOKEN, or --creds-file), exit 1, before the guard ran. These runs used 'creds-file', which is what fleet uses.

One caveat on the fix

The fixed build removes the bootstrap key rather than setting it to null. The guard tests state.bootstrap != null, and loose != treats undefined as null, so it passes. A strict !== null would have failed it. Guard and writer agree today only by virtue of a loose comparison — worth pinning deliberately.

Not merging, and nothing was pushed to fix/455-state-json-single-writer. Khaliq holds the merge gate.

🤖 Generated with Claude Code

https://claude.ai/code/session_01R22c8M6BAYQykoDjVEmMtv

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-02T12:57:05.545719Z	c52d5ae	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: da30ec0f-fe1c-4518-847f-a72e4ad5b43d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c52d5ae8ff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

Relayfile Eval Review

Run: .relayfile/evals/runs/2026-09-02T15-23-09-529Z-HEAD-provider
Mode: provider
Git SHA: 11f638e

Passed: 4 | Needs human: 0 | Reviewable: 0 | Missing output: 0 | Failed: 0 | Skipped: 0

Human Review Cases

No reviewable human-review cases captured Relayfile output.

[cubic-dev-ai]

All reported issues were addressed across 9 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

3 issues found across 5 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="scripts/test-sandbox-initial-sync-readiness-e2e.verdict.test.mjs">

<violation number="1" location="scripts/test-sandbox-initial-sync-readiness-e2e.verdict.test.mjs:58">
P2: The red-check never guards a false green where arm B 'passes' without mirroring anything. `verdict()` returns PROOF_HELD whenever `B.guardExit === 0 && B.bootstrapNull` — it never requires `B.mirroredFiles > 0`. Because the control arm A has already proven the subtree holds >= 2000 files, an arm B that exits 0 with zero (or a handful of) mirrored files is evidence the candidate silently failed to mount, yet the harness would report PROOF HELD and gate the merge. Add a red-check case (and a `B.mirroredFiles > 0` requirement in `verdict`) so a candidate that produced no mirror cannot be read as convergence.</violation>
</file>

<file name="scripts/test-sandbox-initial-sync-readiness-e2e.mjs">

<violation number="1" location="scripts/test-sandbox-initial-sync-readiness-e2e.mjs:370">
P1: When arm B encounters a non-auth network or server error, the harness reports REFUTED instead of UNKNOWN. Require a successful candidate traversal with no infrastructure/error signal before classifying non-convergence as a refutation.</violation>

<violation number="2" location="scripts/test-sandbox-initial-sync-readiness-e2e.mjs:393">
P2: When `RELAYFILE_PROOF_MODULE_BASE` contains a different sandbox package, this harness silently tests its guard as if it were the pinned production guard. Assert the expected package version or resolve dependencies from the pinned lockfile before running the proof.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: When arm B encounters a non-auth network or server error, the harness reports REFUTED instead of UNKNOWN. Require a successful candidate traversal with no infrastructure/error signal before classifying non-convergence as a refutation.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At scripts/test-sandbox-initial-sync-readiness-e2e.mjs, line 370:

<comment>When arm B encounters a non-auth network or server error, the harness reports REFUTED instead of UNKNOWN. Require a successful candidate traversal with no infrastructure/error signal before classifying non-convergence as a refutation.</comment>

<file context>
@@ -187,38 +309,116 @@ function summarize(r) {
+  if (B.stateMissing) {
+    return [EXIT_UNKNOWN, `UNKNOWN: arm B left no state file at ${B.stateFile}`];
+  }
+  if (B.guardExit !== 0 || !B.bootstrapNull) {
+    return [EXIT_PROOF_REFUTED,
+      `REFUTED: arm B exited ${B.guardExit} with bootstrap${B.bootstrapNull ? " null" : " still non-null"} `
</file context>

[cubic-dev-ai]

P2: The red-check never guards a false green where arm B 'passes' without mirroring anything. verdict() returns PROOF_HELD whenever B.guardExit === 0 && B.bootstrapNull — it never requires B.mirroredFiles > 0. Because the control arm A has already proven the subtree holds >= 2000 files, an arm B that exits 0 with zero (or a handful of) mirrored files is evidence the candidate silently failed to mount, yet the harness would report PROOF HELD and gate the merge. Add a red-check case (and a B.mirroredFiles > 0 requirement in verdict) so a candidate that produced no mirror cannot be read as convergence.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At scripts/test-sandbox-initial-sync-readiness-e2e.verdict.test.mjs, line 58:

<comment>The red-check never guards a false green where arm B 'passes' without mirroring anything. `verdict()` returns PROOF_HELD whenever `B.guardExit === 0 && B.bootstrapNull` — it never requires `B.mirroredFiles > 0`. Because the control arm A has already proven the subtree holds >= 2000 files, an arm B that exits 0 with zero (or a handful of) mirrored files is evidence the candidate silently failed to mount, yet the harness would report PROOF HELD and gate the merge. Add a red-check case (and a `B.mirroredFiles > 0` requirement in `verdict`) so a candidate that produced no mirror cannot be read as convergence.</comment>

<file context>
@@ -0,0 +1,134 @@
+const A = (o = {}) => ({ ...goodControl, ...o });
+const B = (o = {}) => ({ ...goodCandidate, ...o });
+
+test("must-fire: the real observed run is a pass", () => {
+  const [code, msg] = verdict(A(), B());
+  assert.equal(code, EXIT_PROOF_HELD);
</file context>

[cubic-dev-ai]

P2: When RELAYFILE_PROOF_MODULE_BASE contains a different sandbox package, this harness silently tests its guard as if it were the pinned production guard. Assert the expected package version or resolve dependencies from the pinned lockfile before running the proof.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At scripts/test-sandbox-initial-sync-readiness-e2e.mjs, line 393:

<comment>When `RELAYFILE_PROOF_MODULE_BASE` contains a different sandbox package, this harness silently tests its guard as if it were the pinned production guard. Assert the expected package version or resolve dependencies from the pinned lockfile before running the proof.</comment>

<file context>
@@ -187,38 +309,116 @@ function summarize(r) {
+      const loader = await externalLoader();
+      disposeResolver = loader.dispose;
+      return Promise.all([
+        loader.load("@agent-relay/sandbox/core"),
+        loader.load("@daytonaio/sdk"),
+        loader.load("@cloud/core/relayfile/client.js"),
</file context>