Fleet does the labor — Slice 1: round-trip proof + prompt hardening (v1.10.0-rc.1)

.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "baton",
   "displayName": "Baton",
-  "version": "1.9.0",
+  "version": "1.10.0-rc.1",
   "description": "Pass the baton. Conduct the fleet. Claude Code as command-and-control for a fleet of coding LLMs — capability routing, cost engine, jobs, decisions, and a knowledge base.",
   "author": { "name": "Kevin Rank", "url": "https://github.com/Ryfter" },
   "repository": "https://github.com/Ryfter/baton",

AGENTS.md

@@ -31,3 +31,6 @@ Active-session detection and resume pointers use a neutral marker contract
 under `$BATON_HOME/sessions/` (`{agent,session_id,cwd,started_at}`). The
 Claude adapter (SessionStart/SessionEnd hooks) ships now; a Codex lifecycle
 adapter writing the same marker shape is the documented follow-on.
+
+Fleet health (model-agnostic):
+- `fleet doctor --live` — harness-neutral way to verify a box's roster actually answers (canary round-trip per enabled provider), not just that the binaries are installed.

commands/fleet.md

@@ -1,6 +1,6 @@
 ---
 description: Manage and invoke the LLM fleet. `doctor` health-checks providers, `test` dispatches a prompt to one provider, `list` shows the registry.
-argument-hint: doctor | test <name> "<prompt>" [--model <m>] | list
+argument-hint: doctor [--live] [--timeout <s>] | test <name> "<prompt>" [--model <m>] | list
 ---
 
 # /baton:fleet
@@ -15,13 +15,13 @@ Operate the fleet defined in `$BATON_HOME/fleet.yaml` (default `~/.baton/fleet.y
 
 2. **Dispatch by subcommand:**
 
-   **`doctor`** — run:
+   **`doctor`** — run (with `--live` to enable canary probes):
 
    ```powershell
-   & pwsh -NoProfile -File "$HOME/.claude/scripts/fleet-doctor.ps1"
+   & pwsh -NoProfile -File "$HOME/.claude/scripts/fleet-doctor.ps1" -Live -TimeoutS 60
    ```
 
-   Echo the table to the user.
+   Echo the table to the user. With `--live`, each enabled provider receives a `PONG` canary and reports `live_ok`, `live_fail(<reason>)`, or `skip`; a `no-canary` reason means the provider ran but didn't answer (often a wrong command template for this box). Plain (no `--live`) performs the fast PATH/reachability check only.
 
    **`list`** — run:
 

docs/superpowers/specs/2026-07-04-fleet-labor-slice1-roundtrip-proof-design.md

@@ -0,0 +1,249 @@
+# Fleet Does the Labor — Slice 1: Round-Trip Proof + Prompt Hardening
+
+**Date:** 2026-07-04 · **Status:** design (approved shape, pending spec review) ·
+**Track:** "Fleet does the labor" (make the conductor farm coding work to non-Claude instruments)
+
+## Problem
+
+Baton's whole thesis is *command-and-control for a fleet of coding LLMs*, but
+today the fleet only demonstrably works with Claude. The plumbing to invoke
+other instruments exists and is real:
+
+- `fleet.yaml` carries a roster of 8 providers with invoke templates
+  (`claude`, `codex exec`, `agy`, `gh copilot`, `ollama`, a remote Ollama box,
+  two LM Studio pins, `gh models`).
+- `Invoke-Fleet` / `Invoke-Fleet-Cli` / the http branch really shell out, with a
+  stdin-safe path for large/quoted prompts.
+- `Select-Capability` routes by capability + cost; the conductor plans a task
+  DAG and routes each task to a chosen provider.
+
+But two things are unproven, and one seam is empty:
+
+1. **No proven end-to-end round-trip.** `fleet doctor` only checks *"is the
+   binary on PATH / is the base_url up."* It never sends a prompt and confirms a
+   coherent answer comes back. The repo `fleet.yaml` is a shared **seed**; the
+   live per-box roster (`~/.baton/fleet.yaml`, box-private) is where real
+   templates live and may be wrong or untested for a given machine.
+2. **Prompt fragility.** The legacy CLI dispatch path interpolates the prompt via
+   `Invoke-Expression` — quote-fragile and subject to the 965-byte argument
+   ceiling. Only `stdin: true` providers survive real coding prompts.
+3. **The executor seam is empty.** `Invoke-TaskViaFleet` is documented
+   *"Non-destructive by construction — it never touches the repo; real
+   code/merge execution is wired by a box via `-Spawner`."* So the conductor
+   routes, calls, records *which* model it chose and whether it exited 0 — then
+   discards the output. Nothing turns a model's work into a repo change.
+
+**This spec covers Slice 1 only:** prove the round-trip and harden the prompt
+path, so that a later slice can build the executor on a pipe that is known to
+work. Slice 1 does **not** apply repo changes, build the executor, or change
+routing. The `-Spawner` executor (gap 3) is Slice 2, a separate spec.
+
+## Goal
+
+A user can run one command and get, per **enabled** instrument on their box, an
+honest verdict: *this model actually answered a real prompt* — or *it didn't,
+and here's why.* And the dispatch pipe is hardened so the real prompts a future
+executor sends won't be mangled.
+
+## Scope & non-goals
+
+**In scope:**
+
+- A live round-trip probe over the enabled roster, surfaced through `fleet doctor`.
+- A deterministic, judge-free pass criterion (a canary token).
+- Making the stdin dispatch path the default for CLI providers so real prompts
+  survive.
+- Plain-English + `--json` legibility of the results.
+- Hermetic tests (fake dispatcher; never touches real CLIs, network, or
+  `~/.baton`).
+
+**Out of scope (later slices / separate specs):**
+
+- The `-Spawner` executor that applies repo changes (Slice 2).
+- Any routing / `Select-Capability` change.
+- Driving Baton *from* Codex/Gemini (the Command Center Codex-adapter follow-on).
+- Per-provider latency benchmarking, sample-output capture, quality scoring.
+
+## Approach (chosen)
+
+**Extend `fleet doctor` with a `--live` probe** rather than adding a new command
+or a `/baton:go` preflight. Doctor already iterates enabled providers and reports
+`ok | skip | err`; the live probe is a second, opt-in pass over the same roster.
+One health surface answers both "is my fleet reachable?" and "does my fleet
+actually answer?"
+
+Rejected alternatives:
+
+- **New `/baton:fleet test` command** — a whole new surface that overlaps
+  doctor's job. A dedicated command is a reasonable *later* affordance once
+  there's more per-provider detail to show; not needed now.
+- **Preflight inside `/baton:go`** — couples proof to execution and taxes every
+  run. Premature.
+
+## Design
+
+### 1. Surface & modes
+
+`scripts/fleet-doctor.ps1` gains a `-Live` switch (slash surface: `--live`) and
+keeps `-Json`.
+
+- **Default (no `-Live`):** today's behavior verbatim — reachability probe
+  (binary on PATH; `base_url` / env-URL reachable). Byte-for-byte unchanged.
+- **`--live`:** for each **enabled** provider, run the reachability check first;
+  if it passes, run a live canary round-trip. Disabled providers are `skip`.
+- Exit code: `0` if every enabled provider is `live_ok`; `1` if any enabled
+  provider fails the live probe. (Matches doctor's existing all-ok/any-bad
+  contract.)
+
+### 2. The canary round-trip contract
+
+A new pure-ish helper (in `fleet-lib.ps1` or a small `fleet-probe-lib.ps1`,
+implementer's call at plan time) sends a fixed canary prompt and classifies the
+result. The probe dispatches through **`Invoke-Fleet`** (not `Invoke-Fleet-Cli`
+directly) so both `kind: cli` and `kind: http` providers — including the local
+LM Studio / Ollama boxes — are covered by the same code path. A `-Dispatcher`
+scriptblock seam is injected for tests.
+
+- **Canary battery (constant):** a set of 4 trivial challenges, dispatched as one
+  combined numbered prompt (a single round-trip). Only the first challenge's
+  answer appears in the prompt text; the other three do **not**, so a template
+  that merely echoes the prompt cannot score them:
+  1. *Reply with the word PONG.* → token `PONG` (literal instruction-following)
+  2. *What is 6 times 7?* → token `42` (echo-proof)
+  3. *What is the capital of France?* → token `PARIS` (echo-proof)
+  4. *What is the opposite of the word HOT?* → token `COLD` (echo-proof)
+- **Score:** count of the 4 tokens present in stdout (case-insensitive substring).
+- **Pass (`live_ok`):** dispatch returns exit 0 **and** all 4 tokens are present
+  (score 4/4) **and** it completed within the probe timeout. A partial score
+  (`live_fail` reason `canary 2/4 (missing: 42, PARIS)`) tells you exactly how a
+  provider misbehaved; an echo-back scores 1/4 and correctly fails.
+- **Fail (`live_fail`):** with a single-word reason:
+  - `not-on-PATH` — reachability check failed (cli binary missing).
+  - `unreachable` — reachability check failed (http base_url / env URL down).
+  - `timeout` — exceeded the probe timeout.
+  - `nonzero-exit` — dispatch exit code ≠ 0.
+  - `no-canary` — exit 0 but stdout lacks the token (e.g. the CLI printed its
+    help text, or the seed template is wrong for this box). **This is the payoff
+    line** — it distinguishes "the pipe/template is wrong" from "the model is
+    down."
+- **Skip (`skip`):** provider disabled in `fleet.yaml`.
+
+**Timeout.** A probe timeout (default 60s; overridable via a `-TimeoutS` param /
+`--timeout`) is measured and enforced. `Invoke-Fleet-Cli` does not currently
+enforce its `TimeoutS` param, so the probe must wrap the dispatch in an enforced
+timeout guard (e.g. a job/async wait) rather than assume the callee honors it;
+http providers already carry `timeout_s`. The probe records elapsed seconds per
+provider for the report.
+
+**Result shape (per provider):**
+
+```
+@{ name; kind; enabled; reachable = $true|$false; live = 'live_ok'|'live_fail'|'skip';
+   reason = <string|null>; elapsed_s = <int>|$null }
+```
+
+### 3. Prompt robustness (gap 2)
+
+Make the **stdin path the default** for `kind: cli` dispatch in
+`Invoke-Fleet-Cli`: when a provider's resolved command is a clean token list
+(exe + args, no shell metacharacters requiring interpolation), pass the prompt
+via the existing temp-file→stdin mechanism instead of interpolating `{{prompt}}`.
+This immunizes real (large, quote-heavy) prompts against the 965-byte ceiling and
+quote mangling — the foundation Slice 2's executor depends on.
+
+- Providers already marked `stdin: true` are unchanged.
+- Providers whose template still *requires* `{{prompt}}` interpolation (a shell
+  form that can't take stdin) keep the legacy path; the change is opportunistic,
+  not forced, so no seed template silently breaks.
+- The canary probe itself **always** uses the stdin path.
+- This must not regress the existing `Invoke-Fleet` cli tests; where a seed
+  template's semantics would change, prefer adding `stdin: true` to that seed
+  entry over rewriting dispatch behavior invisibly.
+
+> **Open implementation note for the plan:** the exact predicate for "clean token
+> list, safe to send via stdin" must be pinned to a concrete, tested rule (e.g.
+> "template has no `{{prompt}}` placeholder AND no shell operators
+> `| > < & ; $(` ") so behavior is deterministic and covered by a unit test. The
+> writing-plans step resolves this to exact code + test cases.
+
+### 4. Legibility
+
+Human report (doctor `--live`), one row per provider, plain English:
+
+```
+PROVIDER           REACHABLE  LIVE        DETAIL
+codex              yes        live_ok     1.2s
+gemini-antigravity yes        live_ok     3.4s
+ollama-local       yes        live_fail   timeout>60s
+gh-copilot         yes        live_fail   no-canary (returned help text, not an answer)
+lm-studio          yes        live_ok     2.1s
+github-models      —          skip        disabled in fleet.yaml
+```
+
+`--json` emits the array of result shapes above for programmatic use (e.g. a
+future dashboard tile).
+
+### 5. Testing (hermetic)
+
+- A **fake `-Dispatcher`** is injected into the probe so the suite never invokes
+  a real CLI, touches the network, or reads real `~/.baton`. The fake returns
+  canned `@{ stdout; stderr; exit_code }` tuples to exercise every branch:
+  `live_ok`, `nonzero-exit`, `no-canary`, `timeout` (simulated), `skip` for
+  disabled, and both `not-on-PATH` / `unreachable` reachability fails.
+- Temp `fleet.yaml` fixtures; temp `BATON_HOME`; `try/finally` restore. Never
+  touch real `~/.baton`, `~/.claude`, `D:\Dev\Grimdex`, or `D:\dev`.
+- Stdin-default dispatch gets unit tests for the "clean token list" predicate
+  (both directions) and a regression assert that `stdin: true` providers and
+  interpolation-required providers are unchanged.
+- The live mode against real box CLIs is a **manual** diagnostic, not part of the
+  automated suite.
+
+### 6. Deploy & docs
+
+- If a new `fleet-probe-lib.ps1` is introduced, add it to the `bootstrap.ps1`
+  deploy manifest **and** add a `test-bootstrap.ps1` deploy assert (the v1.8.0
+  coach-lib omission lesson: every new deployed script gets a deploy assert).
+- `commands/` doc for `fleet doctor` updated to document `--live` / `--timeout`
+  / `--json`.
+- `AGENTS.md`: one line noting `fleet doctor --live` as the model-agnostic way to
+  verify any box's roster actually answers.
+- Plugin version bump (minor) at release.
+
+## House rules (§11, per project standing rules)
+
+- Every shell command arg < 965 bytes; large prompts go via file/stdin.
+- CLI errors: `[Console]::Error.WriteLine(...)` + `exit 2` (never `Write-Error`
+  under `Stop`). Doctor keeps its existing exit-code contract.
+- All file writes `utf8NoBOM`.
+- `ConvertFrom-Json` auto-parses ISO dates to `DateTime` — re-stringify on
+  round-trip. `ConvertTo-Json` needs `-InputObject @(...)` for guaranteed arrays.
+- Never name PS vars `$args/$input/$event/$matches/$host/$pid`.
+- Unary-comma flatten `,([object[]]$x)` only on direct-assignment returns; use
+  `@($x)` when callers pipe.
+- Guard `0/0` NaN in any elapsed/utilization math.
+- Box-private: never write real roster/endpoint values into the shared seed
+  `fleet.yaml`; placeholder hosts only. The live probe reads the box-private
+  live roster at run time.
+
+## Decisions made
+
+- **Prove the round-trip before building the executor** — cheap de-risking slice
+  first; the executor (gap 3) is designed against a pipe known to work.
+- **Extend `fleet doctor --live`** rather than a new command or a `go` preflight —
+  one health surface, minimal new code.
+- **Canary battery pass criterion** (4 challenges scored k/4; `PONG`/`42`/`PARIS`/
+  `COLD`), judge-free — catches help-text / wrong-template / garbage / echo-back
+  responses, not just exit 0. (Revised from a single `PONG` token during review:
+  the token lived inside its own prompt, so an echoing template false-passed.)
+- **Stdin path as the CLI dispatch default** — hardens the pipe for the real
+  prompts Slice 2 will send.
+
+## Out-of-scope follow-ons (named, not built here)
+
+- **Slice 2 — the `-Spawner` executor:** send the task to the chosen instrument,
+  capture its output/edits, turn that into an applied repo change (agentic tools
+  edit in-place; chat models emit a diff Baton applies, or file-edit tasks route
+  only to agentic instruments — resolved in Slice 2's spec), verify via the
+  existing acceptance gate, work on a branch/worktree for reversibility.
+- Per-provider latency/quality benchmarking; sample-output capture.
+- A dashboard tile consuming `fleet doctor --live --json`.

scripts/bootstrap.ps1

@@ -256,7 +256,7 @@ if (-not (Test-Path $scriptsDst)) {
     if ($DryRun) { Write-Ok "[dry-run] would create $scriptsDst" }
     else { New-Item -ItemType Directory -Force -Path $scriptsDst | Out-Null; Write-Ok "created $scriptsDst" }
 }
-foreach ($script in @('baton-home.ps1', 'job-lib.ps1', 'consolidate-lessons.ps1', 'parse-otel.ps1', 'fleet-lib.ps1', 'fleet-doctor.ps1', 'fleet-ensemble.ps1', 'routing-lib.ps1', 'saturation-lib.ps1', 'effective-cost-lib.ps1', 'routing-dispatch.ps1', 'routing-learn.ps1', 'routing-calibrate.ps1', 'routing-cascade.ps1', 'prime-hours.ps1', 'six-hats-lib.ps1', 'council-lib.ps1', 'code-lib.ps1', 'kb-lib.ps1', 'decisions-lib.ps1', 'consolidate-decisions.ps1', 'cost-lib.ps1', 'runs-lib.ps1', 'statusline-feed.ps1', 'fleet-runs-bridge.ps1', 'fleet-orchestrate.ps1', 'fleet-backlog.ps1', 'run-backlog.ps1', 'fleet-models.ps1', 'triage-lib.ps1', 'fleet-triage.ps1', 'usage-lib.ps1', 'fleet-usage.ps1', 'projects-lib.ps1', 'fleet-projects.ps1', 'research-gate-lib.ps1', 'fleet-research-gate.ps1', 'conductor-lib.ps1', 'fleet-go.ps1', 'cost-resolver-lib.ps1', 'prompt-pool-lib.ps1', 'optimize-prompt-lib.ps1', 'fleet-optimize-prompt.ps1', 'memory-lib.ps1', 'fleet-memory.ps1', 'worker-lib.ps1', 'fleet-worker.ps1', 'gate-lib.ps1', 'fleet-gate.ps1', 'fleet-effective-cost.ps1', 'idea-lib.ps1', 'start-lib.ps1', 'coach-lib.ps1', 'session-markers-lib.ps1', 'registry-lib.ps1', 'fleet-project.ps1')) {
+foreach ($script in @('baton-home.ps1', 'job-lib.ps1', 'consolidate-lessons.ps1', 'parse-otel.ps1', 'fleet-lib.ps1', 'fleet-doctor.ps1', 'fleet-ensemble.ps1', 'routing-lib.ps1', 'saturation-lib.ps1', 'effective-cost-lib.ps1', 'routing-dispatch.ps1', 'routing-learn.ps1', 'routing-calibrate.ps1', 'routing-cascade.ps1', 'prime-hours.ps1', 'six-hats-lib.ps1', 'council-lib.ps1', 'code-lib.ps1', 'kb-lib.ps1', 'decisions-lib.ps1', 'consolidate-decisions.ps1', 'cost-lib.ps1', 'runs-lib.ps1', 'statusline-feed.ps1', 'fleet-runs-bridge.ps1', 'fleet-orchestrate.ps1', 'fleet-backlog.ps1', 'run-backlog.ps1', 'fleet-models.ps1', 'triage-lib.ps1', 'fleet-triage.ps1', 'usage-lib.ps1', 'fleet-usage.ps1', 'projects-lib.ps1', 'fleet-projects.ps1', 'research-gate-lib.ps1', 'fleet-research-gate.ps1', 'conductor-lib.ps1', 'fleet-go.ps1', 'cost-resolver-lib.ps1', 'prompt-pool-lib.ps1', 'optimize-prompt-lib.ps1', 'fleet-optimize-prompt.ps1', 'memory-lib.ps1', 'fleet-memory.ps1', 'worker-lib.ps1', 'fleet-worker.ps1', 'gate-lib.ps1', 'fleet-gate.ps1', 'fleet-effective-cost.ps1', 'idea-lib.ps1', 'start-lib.ps1', 'coach-lib.ps1', 'session-markers-lib.ps1', 'registry-lib.ps1', 'fleet-project.ps1', 'fleet-probe-lib.ps1')) {
     $src = Join-Path $repoRoot "scripts\$script"
     $dst = Join-Path $scriptsDst $script
     Copy-WithPrompt $src $dst "lib script: $script" -Force

scripts/fixtures/fleet-sample.yaml

@@ -53,3 +53,13 @@ providers:
     cost_tier: local
     usage_class: tight
     command_template: 'pwsh -NoProfile -Command "Write-Output tight-{{prompt}}"'
+
+  # stdin:true provider — the prompt is piped via stdin, template carries no
+  # {{prompt}}. Guards the regression where the stdin path called
+  # Resolve-FleetCommand with an empty prompt and threw. Echoes stdin back.
+  - name: stub-stdin
+    kind: cli
+    enabled: true
+    cost_tier: local
+    stdin: true
+    command_template: 'pwsh -NoProfile -Command [Console]::In.ReadToEnd()'

scripts/fleet-doctor.ps1

@@ -6,10 +6,13 @@
 #>
 param(
     [string]$Path = $(if ($env:BATON_HOME) { Join-Path $env:BATON_HOME 'fleet.yaml' } else { Join-Path $HOME '.baton/fleet.yaml' }),
-    [switch]$Json
+    [switch]$Json,
+    [switch]$Live,
+    [int]$TimeoutS = 60
 )
 $ErrorActionPreference = 'Stop'
 . (Join-Path $PSScriptRoot 'fleet-lib.ps1')
+if ($Live) { . (Join-Path $PSScriptRoot 'fleet-probe-lib.ps1') }
 
 try {
     $fleet = Read-Fleet -Path $Path
@@ -21,6 +24,29 @@ try {
     Write-Host "fleet doctor: $($_.Exception.Message)" -ForegroundColor Red
     exit 1
 }
+
+if ($Live) {
+    $results = foreach ($p in $fleet) { Invoke-FleetProbe -Provider ([hashtable]$p) -TimeoutS $TimeoutS -FleetPath $Path }
+    $rows = @($results)
+    if ($Json) {
+        ConvertTo-Json -InputObject @($rows) -Depth 4
+    } else {
+        $render = $rows | ForEach-Object {
+            $reach = if ($null -eq $_.reachable) { '-' } elseif ($_.reachable) { 'yes' } else { 'no' }
+            $detail = if ($_.reason) { $_.reason }
+                      elseif ($null -ne $_.score -and $null -ne $_.elapsed_s) { "$($_.score)/$($script:FleetCanaryChallenges.Count)`u{00B7}$($_.elapsed_s)s" }
+                      elseif ($null -ne $_.elapsed_s) { "$($_.elapsed_s)s" }
+                      else { '' }
+            [pscustomobject]@{ PROVIDER = $_.name; REACHABLE = $reach; LIVE = $_.live; DETAIL = $detail }
+        }
+        $render | Format-Table PROVIDER, REACHABLE, LIVE, DETAIL -AutoSize | Out-String | Write-Host
+        $enabled = @($fleet | Where-Object { $_.enabled -eq $true }).Count
+        Write-Host "$enabled enabled provider(s); live round-trip."
+    }
+    $anyLiveBad = @($rows | Where-Object { $_.enabled -eq $true -and $_.live -ne 'live_ok' }).Count -gt 0
+    if ($anyLiveBad) { exit 1 } else { exit 0 }
+}
+
 $rows = @()
 $anyBad = $false