feat(ce-optimize): remote experiment backend with paired self-measurement

[kieranklaassen]

Summary

Design C of the approved ce-optimize long-runs brainstorm, stacked on #1707 (eval discipline), which is stacked on #1708 (Design B). Adds execution.backend: remote (Q7): one experiment per detached worker with its own checkout, collected asynchronously through B's tick/receipt model. Nothing about what counts as an improvement changes; decide.mjs still makes every decision.

Third capability condition in SKILL.md ## Execution Surface: a detached worker is a dispatch whose result arrives as a pushed ref, a store file, or a host-delivered message, not as edits in the local tree. remote needs one; without it that spec runs on worktree. No vendor names in the body; the one example table in persistence.md gains a column.

Worker contract (Q2). experiment-prompt-template.md gets one delta block appended only for remote: verify HEAD is {base_sha}, implement inside scope.mutable, measure paired (baseline commit and candidate on the same machine, alternating samples), write result.yaml (base_sha, head_sha, machine, both snapshots, correctness; per-case cases and cost fields only when the harness emits them), commit and push optimize-exp/<spec>/exp-NNN, report a structured result. The rest of the template is unchanged.

Orchestrator side (loop.md).

• 3.2: remote row in the dispatch table; isolation is the worker's checkout, so the recipe records base_sha and pushes it. Falls back to the worktree row for the whole run when no such capability is in the tool list.
• 3.3: a remote result is accepted only when base_sha matches what was dispatched, the range touches only scope.mutable + result.yaml, and both snapshots carry the required objectives; otherwise error, never repaired. The worker's pair is the selection measurement; result.yaml is copied into <state-root> as the recovery marker. A worker result carries only the selection pairing, so a configured holdout answers a would-be keep with next_measurement: holdout like any other backend.
• 3.4: one independent measurement owns both the confirmation and the holdout for a remote winner: a pairing the candidate's author did not produce (this checkout or a fresh confirmation worker), at the full protocol, which also runs the holdout when the last decision asked for it. Persisted as kind: standalone and kind: holdout pairings, each with machine and measured_by; decide.mjs runs on those snapshots, never the worker's. result.yaml is stripped from what merges.

Schema. backend enum + remote; rule: remote requires comparison.method paired or relative (cross-machine absolute numbers are not comparable); max_concurrent note per backend. Log schema: comparisons[].machine and comparisons[].measured_by on every pairing kind including holdout, plus a result_marker shape. measurement.md: 1.4/1.5 apply only when experiments share this machine.

Files

• skills/ce-optimize/SKILL.md (7949 bytes CRLF-adjusted; a few sentences compacted to fit the third condition under the 8000 Codex bound, none of them pinned behavior)
• references/{loop,measurement,persistence}.md, references/experiment-prompt-template.md, both schemas
• tests/skills/ce-optimize-decide.test.ts – one new pin test (enum value, validation rule, body names the third condition, template delta present and instruction 4 of the base template intact, collection/confirmation/holdout routing conditions, log-schema fields)
• docs/guides/ce-optimize.md – remote paragraph in Reference; FAQ on cross-machine comparability

Design choices the doc left open (closest to today's behavior)

• A worker that reports a mismatched base_sha or a range outside mutable scope is an error and closed, not retried automatically.
• Extra selection samples (add_sample, confirm) for a remote candidate re-dispatch to the same worker (or a new one from the same head_sha); the holdout never goes to the worker. The log entry stays one experiment.
• Result refs may be deleted only after the entry is verified in the log and the result.yaml copy is in the state root.
• scripts/experiment-worktree.sh untouched; no fetch-exp helper was needed.

Rebase onto #1707

Conflicts in loop.md 3.4 (eval's holdout-before-keep bullet vs. the remote confirmation bullet) and the guide FAQ were resolved by keeping the eval blocks verbatim and restating the remote bullet so the independent confirmation is the single owner of the holdout measurement. No eval-discipline content was rewritten.

Validation

• bun install, bun run release:validate (35 skills, in sync).
• bun run test after the rebase: 4120 pass, 2 fail – both subprocess timeouts in unrelated files (ce-babysit-pr-snapshot, cli install-from-GitHub) that pass when run alone (205 pass / 0 fail).
• Behavioral fresh-agent eval skipped.

Security Disclosure

No new shell in the skill. The remote worker prompt instructs the worker to push only to optimize-exp/<spec>/exp-NNN and never to the optimization branch; the orchestrator validates a returned result by base_sha match and a mutable-scope-only commit range before using any number, and requires an independently produced confirmation (and holdout, when configured) measurement before merging. Worker-supplied result.yaml is treated as a claim, copied into the state root, and never edited. Residual risk: a worker can still report false numbers for an exploratory pair; that only costs one confirmation measurement, since the keep decision is not taken from the worker's pair alone.

Agent Disclosure

• Model: Cursor cloud agent · Claude Fable 5.1 (as stated by the harness context; no version beyond that is visible)