feat(ai): teach AI agents to use devstack via MCP, skills and embedded docs

[gustavobertoi]

Why

An AI coding agent dropped into a devstack workspace recognises Docker, does not
recognise devstack, and reaches for docker compose up or hand-writes a
docker-compose.yaml. Both are wrong here — files under .devstack/ are
generated and get overwritten, and the compose project name, labels and external
network are tool-owned, so driving compose directly forks a parallel stack. The
same agent puts a [[ ]] action in a template's description: (a hard lint
error, since metadata keys are parsed unrendered) or writes ${ref.foo} instead
of ${ref:foo}.

Everything needed to prevent that already existed — template list --json is a
catalog, Describe reads metadata without rendering, every headline command has
--json, and there are 9,400 lines of documentation. What was missing was
discovery.

What this adds

devstack ai — one corpus, three surfaces, nothing re-authored.

Command	Does
ai docs [slug]	Read/search the documentation corpus, now compiled into the binary
ai commands	The command tree as data, derived from the live cobra tree
ai mcp	Serve MCP over stdio — 25 tools, 5 prompts, 71 resources
ai install / ai check	Emit (and drift-check) the skills, AGENTS.md block and MCP registration
config schema	The published JSON Schema for devstack.yaml / workspace.yaml

Three design choices carry the weight:

The MCP tools are the CLI. Each handler builds a fresh command tree with
--json and captures stdout. No parallel API to drift, tool semantics match the
CLI by construction, and the flock is taken and released inside each call — so
the long-lived server never holds it and the no-daemon architecture is preserved.

Emitted files are workspace- and version-independent. They describe devstack,
never a snapshot of the project list or the binary's version stamp. Committed
files never go stale on a release or when someone adds a project. Live facts come
from status --json, which the agent runs.

Three merge modes, because two targets are user-owned. internal/ide's
whole-file write would destroy hand-written content. Skills are owned outright;
AGENTS.md/CLAUDE.md use the marker-fence idiom already in production in
internal/dns for /etc/hosts; .mcp.json sets one key and compares it
semantically, so another tool's formatting survives.

Safety

Write tools are on by default, but irreversible verbs (workspace destroy,
db drop, db reset) are absent unless --allow-destructive — MCP has no
TTY, so a mutating tool must inject --yes, and absence is a guarantee where an
annotation is only a hint. The secrets group is never registered at any setting.

Defects found and fixed along the way

• A live 404. internal/ide emitted a $schema URL to
  schemas/devstack.schema.json — a path that existed at no tag. Every
  .vscode/settings.json devstack has ever generated pointed at nothing. It also
  mapped workspace.yaml to the project schema.
• Two dangling doc links (../CLAUDE.md from docs/specs/ resolves to
  docs/CLAUDE.md). All 1,188 relative links are now checked.
• command-reference.md was missing shell-init and others; now asserted
  against the cobra tree in both directions.
• CI skipped doc-only PRs — untenable once docs compile into the binary.
• ai install could write into $HOME. Workspace discovery walks up, so
  one stray workspace.yaml in a home directory scattered AGENTS.md,
  CLAUDE.md and .mcp.json there and installed skills machine-wide. Caught
  while dogfooding; now refused, with two regression tests.
• .mcp.json drift on cosmetic reformats (second commit) — also caught by
  the new gate, during this PR's own dogfood commit.

Testing

111 new tests. Notable ones:

• MCP server driven over the SDK's in-memory transports with a fake Deps — no
  Docker, no subprocess. The tool set is pinned per option mode, so changing
  what a model can do to someone's machine is a test-breaking act.
• An e2e test drives the real binary as an MCP server and parses the entire
  session as JSON-RPC — stdout purity is the most common way a Go MCP server
  ships broken.
• Schema/struct parity by reflection in both directions, plus a fixture
  round-trip through the schema and config.Load.
• devstack dogfoods its own emitter: this repo commits what ai install writes,
  and make ai-check is now part of make ci.

Verified: make ci, make determinism, make smoke, go test -tags=e2e.

One pre-existing failure is unrelated and untouched: TestRunUninstallSequence
fails on a clean main too (verified via git stash) because it doesn't
isolate XDG_BIN_HOME and picks up a real ~/.local/bin/ds symlink.

Cost

Binary grows 2.6 MiB stripped (45.2 → 47.9 MiB): ~845 KiB embedded content,
~1.8 MiB MCP SDK. Deliberately not gated behind a build tag — the emitted
.mcp.json points at devstack ai mcp, so a build without it would ship a
broken registration.

New dependency github.com/modelcontextprotocol/go-sdk (pure Go, CGO_ENABLED=0
holds) is pinned, wrapped by internal/mcpserve as its single seam, and added to
the DECISIONS dependency-risk register.

Docs

Spec 32 · DECISIONS D20 · guide page 21 (ai-agents.md) · new Q-AI-SCOPE
and Q-AI-PLUGIN open questions (global install and a Claude Code plugin are
both deliberately deferred, with reasoning).

🤖 Generated with Claude Code