Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,7 @@ The primary session identity is **Skywalker** (`buildChatRole` → `createSkywal

- `buildChatRole` — Skywalker primary identity (orchestrate; DIY tiny/bounded product edits; spawn for substantial work).
- `buildHarnessFacts` — the non-derivable rules: shell file-writes are blocked, path tools are the DIY surface on primary (spawn builder/docs directors for substantial work), dependency installs and off-limits paths need approval, images are native multimodal input, only core tools are resident (load the rest via `tool_search`; use `search_agents` before dispatching specialists), workflows run only from slash-command steps, and session memory lives at `.corbits/MEMORY.md`.
- `buildGuidelines` — be concise, prefer `spawn_agent`/`wait_agents` for substantial product work, DIY tiny/bounded edits on the parent, answer questions and diagnose visual/product feedback before editing, work autonomously for explicit coding tasks, use `lsp` for symbol work, and verify changes when practical.
- `buildGuidelines` — be concise, prefer `spawn_agent`/`wait_agents` for substantial product work, DIY tiny/bounded edits on the parent, answer questions and diagnose visual/product feedback before editing, work autonomously for explicit coding tasks, use `lsp` for symbol work, and require implementation agents to run the repository-defined typecheck, relevant tests, and every defined full verification command. Agents report exact commands, outcomes, and exit statuses; a repository with no typecheck command produces an explicit Blocker backed by project-configuration evidence rather than an invented command or silent skip.
- `buildPromptDisciplineBlock` — a shared, prohibition-form section appended exactly once to every built prompt (chat and sub-agent, every provider family). Primary vs leaf wording differs for product writes: leaves are told to use `read_file`/`edit_file`/`write_file`; Skywalker is told to DIY tiny/bounded edits with those path tools and spawn directors for substantial work. Shared rules: never `cat`/`sed`/heredoc/`echo` for file work, no setting or exporting environment variables (recurring needs belong in project settings), `web_fetch`/`web_search` instead of `curl`/`wget`/hand-rolled queries, one operation per `run_shell` call, turn semantics (a tool-less reply is the final answer, no repeat searches, stop and change approach after three failed attempts, batch independent reads in parallel), and TTY output rules (short bold headers, one-line bullets, backticks for paths/commands, no wide tables).

**Provider-conditional residuals.** Per-family additions layer on top of the shared block via the same `ModelFamilyPolicy` mechanism the directors use (`src/subagent/provider-family.ts`, `src/agent/model-family-policy.ts`) — additive lines, never prompt forks. **Grok** leaves get `buildGrokLeafAntiThrashNote` (gated by `shouldApplyGrokAntiThrash` / `applyGrokFinishBias`, withheld from orchestrators): a compact finish-bias reinforcement plus a one-line reminder to route file/web work through the dedicated tools rather than `run_shell`, motivated by observed tool-routing thrash on the same harness. **Kimi** intentionally has no residual yet — `detectModelFamily` already resolves the family so callers can branch on it, but the prompt seam is left unfilled pending eval characterization of Kimi's behavior, mirroring the provisional (permissive-default) policy in `model-family-policy.ts`.
Expand Down Expand Up @@ -482,7 +482,7 @@ CLI argv

```

There is no post-submit `build`/`typecheck`/`test` critique step in the current tree; validation is operator- and hook-driven (`postTurn`/`postRun`) plus explicit `run_shell` during agent work.
There is no runtime post-submit `build`/`typecheck`/`test` enforcement in the current tree. Instead, the prompt contract makes every defined typecheck, relevant test command, and defined full verification gate mandatory during implementation work. Reports must carry each exact command, outcome, and exit status; absence of a repository-defined typecheck is an explicit Blocker supported by evidence from project configuration. Operator and lifecycle hooks (`postTurn`/`postRun`) supplement rather than replace that evidence.

## Design Decisions

Expand Down
7 changes: 7 additions & 0 deletions src/agent/directors/builder/package.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,13 @@ describe("builderPackage", () => {
expect(p).toMatch(/Don't shortcut verify/i);
expect(p).toMatch(/partial gates/i);
expect(p).toMatch(/pre-existing/i);
expect(p).toMatch(/defined typecheck command.*relevant tests.*defined full check/is);
expect(p).toMatch(/repository defines no typecheck command.*explicit Blocker/is);
expect(p).toMatch(/evidence.*AGENTS.*package scripts/is);
expect(p).toMatch(/do not invent.*typecheck command/i);
expect(p).toMatch(/exact verification command.*outcome.*exit status/is);
expect(p).toMatch(/bare .*pass.*incomplete report/is);
expect(p).toMatch(/never silently skip/i);
});

test("systemPrompt requires style and philosophy prerequisites", () => {
Expand Down
11 changes: 7 additions & 4 deletions src/agent/directors/builder/package.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,16 @@ Keep the scope tight to the brief. If you discover additional work is needed, fi

## Build Gate

Run the project's full check (\`bun run check\` or the gate the brief / AGENTS.md specifies).
For implementation work, run the repository-defined typecheck command and relevant tests, then run the defined full check (\`bun run check\` or the gate the brief / AGENTS.md specifies). All defined checks are mandatory.

- If the check passes, proceed to report (or commit only if the brief's success_criteria explicitly require it)
- If the check fails due to your changes, fix the failures and re-run until it passes
- If the check fails due to pre-existing issues unrelated to your changes, report under Blockers for the parent; do not silently expand scope
- If the repository defines no typecheck command, do not invent a typecheck command: report its absence as an explicit Blocker with evidence from AGENTS.md and package scripts (or equivalent project configuration)
- If the checks pass, proceed to report (or commit only if the brief's success_criteria explicitly require it)
- If a check fails due to your changes, fix the failures and re-run until it passes
- If a check fails due to pre-existing issues unrelated to your changes, report under Blockers for the parent; do not silently expand scope
- Do not move forward with a broken build you caused
- Do not substitute partial gates (e.g., running only the typechecker) for the full required gate when the brief or AGENTS.md says full check
- In Findings, report every exact verification command and its outcome, including exit status; a bare \`pass\` without command evidence is an incomplete report
- If a check genuinely cannot run because of a missing runtime or dependency, sandbox restriction, or permissions, record the exact inability under Blockers; never silently skip a required check

## Guidelines

Expand Down
20 changes: 17 additions & 3 deletions src/agent/directors/skywalker/package.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -164,13 +164,26 @@ describe("skywalkerPackage", () => {
expect(p).toContain("implement success_criteria");
});

test("systemPrompt has critic-after-implement verify path", () => {
test("systemPrompt requires critic after every builder implementation", () => {
const p = skywalkerPackage.systemPrompt;
expect(p).toContain("Verify after ship");
expect(p).toContain("public-API");
expect(p).toContain("critic");
expect(p).toContain("tester");
expect(p).toContain("correctness/brief gaps");
expect(p).toMatch(/after every delegated \*\*builder\*\* implementation.*run \*\*critic\*\*/is);
expect(p).toMatch(
/substantial implementation limited to one internal file.*still requires Critic/is,
);
expect(p).toMatch(/Builder self-report.*never sufficient to skip/is);
expect(p).toMatch(
/After every delegated builder landing.*run a critic.*architecture.*add greybeard/is,
);
expect(p).not.toMatch(/critic \(or greybeard when architecture is in play\)/i);
expect(p).toMatch(
/Skip a new Critic dispatch only for parent-DIY work or when existing independent review evidence already covers both the resulting diff and its success criteria/i,
);
expect(p).not.toMatch(/Multi-file or public-API changes: after builder/i);
expect(p).not.toMatch(/After multi-file builder landings/i);
expect(p).not.toMatch(/self-report is thin/i);
});

test("systemPrompt spawn-target for substantial code is builder, not implement", () => {
Expand All @@ -188,6 +201,7 @@ describe("skywalkerPackage", () => {
const p = skywalkerPackage.systemPrompt;
expect(p).toContain("blocking");
expect(p).toContain("re-dispatch **builder**");
expect(p).toMatch(/narrowed or changed follow-up brief/i);
expect(p).toContain("ship → verify → fix → re-verify");
expect(p).toContain("Cap re-fix rounds");
});
Expand Down
6 changes: 3 additions & 3 deletions src/agent/directors/skywalker/package.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Quick routing:
- gaasbot = risk counsel
- bruckheimer = product discovery docs
- intern = exact shell / mechanical ops
- After multi-file builder landingsdefault a critic (or greybeard when architecture is in play) on the diff/criteria in a fresh context
- After every delegated builder landingrun a critic on the diff/criteria in a fresh context; when architecture is in play, add greybeard for architecture judgment

Prefer typed spawn: intent, success_criteria, do_not, report_focus, agent when specialist.
Parallelize independent lanes with spawn_agent, then wait_agents. manage_tasks for your checklist. ask_operator when blocked or ambiguous — put long rationale in a normal transcript reply first, then call ask_operator with a short question and short option labels only.
Expand Down Expand Up @@ -88,8 +88,8 @@ When the operator brief states a function signature or return shape, put that **

# Verify after ship

Multi-file or public-API changes: after builder, run **critic** focused on brief + public API contract (sync/async, signatures). Prefer **tester** when you need independent suite evidence and builder's self-report is thin.
If critic (or tester) reports **blocking** findings: re-dispatch **builder** with those findings in success_criteria/do_not — do not declare done on a "ready" that ignored blockers.
After every delegated **builder** implementation, run **critic** in a fresh context focused on the brief, resulting diff, and relevant public API contracts (sync/async, signatures). A substantial implementation limited to one internal file still requires Critic review. Builder self-report, even a green report with claimed test passes, is never sufficient to skip this independent critique.
Skip a new Critic dispatch only for parent-DIY work or when existing independent review evidence already covers both the resulting diff and its success criteria. Use **tester** when you need independent suite evidence. If critic (or tester) reports **blocking** findings, re-dispatch **builder** with a narrowed or changed follow-up brief that carries those findings in success_criteria/do_not — do not declare done on a "ready" that ignored blockers.
Close the loop: ship → verify → fix → re-verify. Cap re-fix rounds (e.g. 1–2) then report Blockers.
Critic flags correctness/brief gaps only — not over-engineering theater.

Expand Down
28 changes: 28 additions & 0 deletions src/agent/prompts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,19 @@ function countOccurrences(haystack: string, needle: string): number {
return haystack.split(needle).length - 1;
}

function expectVerificationGuidance(prompt: string): void {
expect(prompt).toMatch(
/defined typecheck command.*relevant tests.*defined full verification command/is,
);
expect(prompt).toMatch(/repository defines no typecheck command.*explicit Blocker/is);
expect(prompt).toMatch(/evidence.*AGENTS.*package scripts/is);
expect(prompt).toMatch(/do not invent.*typecheck command/i);
expect(prompt).toMatch(/exact verification command.*outcome.*exit status/is);
expect(prompt).toMatch(/bare .*pass.*incomplete report/is);
expect(prompt).toMatch(/never silently skip/i);
expect(prompt).not.toMatch(/relevant checks .*when practical/i);
}

describe("buildPromptDisciplineBlock", () => {
it("references only tool names that exist in the registration source", () => {
for (const name of REFERENCED_TOOL_NAMES) {
Expand Down Expand Up @@ -102,6 +115,21 @@ describe("shared discipline block appears exactly once per built prompt", () =>
});
});

describe("shared verification guidance", () => {
it("requires evidence-carrying verification in worker prompts", () => {
const prompt = buildSubAgentSystemPrompt(undefined, undefined, undefined, {
orchestrator: false,
grokAntiThrash: false,
});
expectVerificationGuidance(prompt);
});

it("requires evidence-carrying verification in orchestrator chat prompts", () => {
const prompt = buildChatSystemPrompt(undefined, undefined, undefined, [], "orchestrator");
expectVerificationGuidance(prompt);
});
});

describe("grok finish-bias residual gating (extends existing provider-family tests)", () => {
it("is present for a grok worker", () => {
const prompt = buildSubAgentSystemPrompt(undefined, undefined, undefined, {
Expand Down
5 changes: 4 additions & 1 deletion src/agent/prompts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,10 @@ export function buildGuidelines(
"- Touch only code required for the task; no drive-by refactors, formatting sweeps, or unrelated fixes.",
"- Follow AGENTS.md and /docs for architecture; load the style and philosophy skills when starting repo work.",
"- Match existing project patterns (functional style, arktype at boundaries, small focused diffs).",
"- Before finishing a code change, run relevant checks (typecheck, tests) when practical.",
"- Before finishing implementation work, run the repository-defined typecheck command, relevant tests, and every defined full verification command; these checks are mandatory.",
"- If the repository defines no typecheck command, do not invent a typecheck command: report its absence as an explicit Blocker with evidence from AGENTS.md and package scripts (or equivalent project configuration).",
"- In Findings, report every exact verification command and its outcome, including exit status. A bare `pass` without command evidence is an incomplete report.",
"- If a required check genuinely cannot run because of a missing runtime or dependency, sandbox restriction, or permissions, record the exact inability under Blockers; never silently skip a required check.",
...(subAgent
? []
: [
Expand Down
Loading