Skip to content

[DOCS] Generate the reference section from committed CLI and REST artifacts - #443

Open
justin13888 wants to merge 9 commits into
chore/freeze-capsule-core-api-399from
docs/reference-generation-415
Open

[DOCS] Generate the reference section from committed CLI and REST artifacts#443
justin13888 wants to merge 9 commits into
chore/freeze-capsule-core-api-399from
docs/reference-generation-415

Conversation

@justin13888

@justin13888 justin13888 commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Description

Publishes /reference/ for the two developer surfaces that have a committed, drift-gated
description artifact: the CLI (S-Z8) and the REST contract (S-Z9). S-Z10
(SDK / FFI / WASM) is filed as #445 with the evidence for why it cannot ship here.

Stacked on chore/freeze-capsule-core-api-399 (PR #426) and targeted at it.

Summary

design/developer-docs.md commits to one docs site with generated reference pages and fixes
the boundary at artifacts, not toolchains: the CI docs job installs bun and nothing
else, so the documentation build reads committed description artifacts and never invokes
cargo. Before this change /reference/ held one page saying nothing was published there yet.

The CLI now has a description artifact. capsule_cli::cli::command_tree() walks the
clap tree built from compile-time attributes and returns JSON; gen_cli_surface writes it
to capsule-cli/cli-surface.json and, with --check, fails when the committed copy is
stale. It mirrors capsule-server/src/bin/gen_openapi.rs argument for argument, and
cli-surface-check joins check-rust beside openapi-check-kynos. No new dependency:
serde_json and clap were already there.

The artifact is deterministic and locale-independent, because a byte-compared committed file
cannot depend on the machine that emits it. Subcommands sort by name so reordering an enum
variant cannot churn the file; arguments keep declaration order, which for a positional is
its position. Command::build is not called, so clap's synthesized --help is not described
sixteen times over and a boolean flag is not documented as accepting true or false.

REST already had onecapsule-server/openapi.json, OpenAPI 3.2, 51 paths and 59
operations, emitted and gated. This change renders it.

One generator turns both into pages. capsule-docs/scripts/gen-reference.mjs runs on
node: builtins alone, is invoked from package.json's dev/build scripts, and adds no
dependency to package.json or bun.lock. Its output is ordinary content-collection
entries, so Pagefind indexes them, starlight-links-validator checks their anchors, the
PageTitle override renders their badge, and the notranslate pass marks up their terms —
everything an embedded OpenAPI application would have forfeited.

The pages are gitignored. A committed copy of generated output is a second source of
truth that can disagree with the artifact it came from, and rule 2 exists so it cannot.
Hand-written prose stays in one overview per surface (reference/cli.md,
reference/api.md), sitting beside the generated directory rather than inside it.

capsule-docs/scripts/reference-groups.mjs is the single ordered page table.
astro.config.mjs builds the Reference sidebar from it and the generator decides which
pages exist from it, so the sidebar stays hand-curated as the design doc requires while a
page with no navigation entry — or navigation with no page — stops being expressible. The
document carries no tags on any of its 59 operations, so there was nothing to autogenerate
a grouping from; the eleven groups track the surface map in design/api-surfaces.md.
An operation matching no group fails the build, naming it.

Three failures are fatal rather than degraded, because the design doc puts a stale reference
page above a missing one: a missing or unparseable artifact, an artifact declaring a schema
or OpenAPI version the generator was not written against, and an ungrouped operation. Nothing
emits a stub.

The docs CI path filter now names every artifact the build reads, not just the site.

Validation

Every command run inside the worktree, on the final head. No failure was caused by this
change.

Command Outcome
mise run cli-surface-check pass
mise run cli-surface-check after hand-editing cli-surface.json (dry_rundryrun) fails as designed, naming mise run cli-surface; passes again once restored
mise run check-docs pass — format, lint, 138 tests, 73 pages built (59 before this change), starlight-links-validator green across every generated route and anchor. Run three consecutive times at an earlier head to prove the parallel-generation race gone.
mise run check-docs-truth pass — cross-links 473/473, endpoint-census 88/88, module-paths 118/118
mise run check-md pass — 0 issues in 172 files
mise run test-docs pass — 138 tests across 6 files, 73 of them in gen-reference.test.mjs
mise run i18n-guard pass
cargo nextest run -p capsule-cli pass — 36 tests, 13 of them over command_tree()
mise run check-rust pass — all 16 steps, run individually: fmt, clippy, doc-check-rust, i18n-check, i18n-guard, openapi-check-kynos, cli-surface-check, architecture-check, license-check, translate-readme-check, build-rust, build-check-wasm, build-ffi, lint-check-ffi, gen-bindings, verify-examples

The build emits 12 generated pages: one CLI page and eleven REST group pages.

check-docs was run three times in a row deliberately. test-docs and build-docs run in
parallel under it, and the repo-level generator test used to render into the working tree
while the Astro build read it — an intermittent failure visible only under the gate. The
test now renders into a temp root seeded from the committed artifacts, and the three clean
runs are the evidence that the race is gone.

Each new guard was shown to be load-bearing by reverting it and observing the failure,
rather than by assuming the test would have caught it:

  • emptying NEVER_SCANNED makes i18n-guard report all three of the emitter's lines;
  • restoring the $ref bound to 2 loses WireBlobRole from /reference/api/sync/;
  • restoring the plain visited set loses a model that only a shallower second path reaches;
  • restoring {2,} to the setext underline loses both single-character cases;
  • restoring the root-only composition scan loses all three nesting cases.

Two further checks were run in a condition CI reproduces and a developer machine does not:

  • mise run check-docs-truth on a tree with no generated pages present (the CI
    condition — the docs-truth job checks out but never builds). Identical result. This was
    the specific hazard the walk exclusion was added for, and it is the only way to observe
    it.
  • Every table row in all twelve generated pages was checked for column-count consistency
    against its header, and every intra-page anchor against the headings that answer it. Zero
    malformed rows, zero dangling anchors. The row audit is what caught the unescaped | in
    string | null, which had silently opened a fourth column in the real output.

gen-bindings and verify-examples each failed once with No space left on device and
passed on retry. The cause was the machine, not the change: the 2.8 TB scratch volume shared
by this run's lane worktrees was at 100%, and the failures were cargo unable to write
libcapsule_core.a and an .rmeta. Both passed after deleting this worktree's own
regenerable output (target/incremental, target/doc); no other worktree was touched.
Independently, CI's Rust (fmt + clippy + build) job — which runs mise run check-rust, the
same 16 steps, on a runner with its own disk — is green on this head, as is Rust (tests).

Not proven, stated rather than papered over: no gate proves a rendered page is
semantically faithful to its artifact. The generator's unit tests over fixtures shaped
like the real documents are the substitute, and that is weaker.

Risks and rollout

  • No gate proves a rendered page is semantically faithful to its artifact. The
    generator's unit tests over fixtures are the substitute, and that is a weaker guarantee.
    Stated rather than papered over; it is the reason the fixtures are shaped like the real
    documents (an operation description that opens at #, a nullable union, a pipe inside a
    description).
  • Three files change shared gates: mise.toml (check-rust gains
    cli-surface-check), .github/workflows/ci.yml (two path filters), and
    capsule-docs/scripts/lib/walk.mjs (the docs-truth walk prunes the generated
    directories). Each is additive; none narrows an existing gate.
  • Every slice reverts standalone. Reverting the docs commits leaves an unread committed
    artifact and a check-rust entry that still passes; reverting the CLI commit as well
    leaves the tree as it was.
  • capsule-cli gains a second binary. release.yml copies capsule by name, so
    packaging is unchanged; cargo build -p capsule-cli --release compiles one extra small
    binary per target.

Related Issues

Refs #415S-Z8 and S-Z9 land here. S-Z10 remains and is filed as #445.

Contributor Checklist

  • I agree to the Contributor License Agreement for this and future contributions.
  • My code follows the project's style guidelines according to CONTRIBUTING.md.
  • Tests pass
  • No sensitive info / secrets
  • Docs updated if needed

Decisions taken

Issue 415 - docs: generate the reference section — CLI, REST, and SDK/FFI/WASM (S-Z8, S-Z9, S-Z10)
Plan:     v1 (planned against f433d918; executed on the head of lane #399's branch), amended by the orchestrator for the developer-docs.md hunk
Branch:   docs/reference-generation-415
Base:     chore/freeze-capsule-core-api-399 (head of PR #426), stacked; the PR targets that branch
Worktree: /var/mnt/scratch/golem/dev/Capsulsaurus/Capsule.worktrees/Capsule-docs-reference-generation-415
Cause:    -
Touches:  capsule-cli/src/bin/gen_cli_surface.rs (new), capsule-cli/cli-surface.json (new, committed artifact), capsule-cli/src/cli/mod.rs (pub fn command_tree(); pinned to Bundle::for_locale("en") so lane #413's localisation cannot perturb the artifact), capsule-cli/Cargo.toml ([[bin]]), mise.toml (cli-surface, cli-surface-check, one check-rust entry after openapi-check-kynos), .github/workflows/ci.yml (docs + docs-truth path filters += capsule-cli/cli-surface.json, capsule-server/openapi.json), .markdownlint-cli2.jsonc (ignore generated reference), capsule-docs/package.json (dev/build scripts prefixed with the generator), capsule-docs/.gitignore, capsule-docs/astro.config.mjs (Reference sidebar from reference-groups), capsule-docs/scripts/{gen-reference.mjs, gen-reference.test.mjs, reference-groups.mjs} (new), capsule-docs/scripts/docs-truth.mjs (walk exclusion for generated pages), capsule-docs/src/content/docs/reference/{index.md, cli/index.md (new), api/index.md (new)}, capsule-docs/src/content/docs/design/developer-docs.md (ONLY the surface-status table rows for REST and CLI → Landed; NOT the "REST is blocked" paragraph, which lane #398 rewrites), SLICES.md (rows/blocks S-Z8, S-Z9, S-Z10 ONLY)
Will not: touch capsule-server/**, capsule-sdk/**, capsule-wasm/**, capsule-web/**, mise-tasks/gen-bindings, Cargo.toml, bun.lock; add any dependency to capsule-docs/package.json or Cargo.toml
Lane:     serialised behind #399 (mise.toml check-rust list; capsule-cli). Forecast collisions: mise.toml check-rust list with #399 (adjacent entries), .github/workflows/ci.yml filters with #416/#400, developer-docs.md with #398 (distinct hunks by construction). Lane #413 (CLI) stacks on this lane's head.
Settled:  Base = head of PR #418 → stacks on #399. Route naming /reference/api/ per developer-docs.md:145-155.
  1. Deliverable boundary — ship S-Z8 (CLI) and S-Z9 (REST); file S-Z10 (SDK/FFI/WASM).
    Taken: CLI + REST now; no manifest entry under capsule-sdk, capsule-core-ffi, capsule-wasm or mise-tasks/gen-bindings.
    Rejected: All three surfaces - uniffi 0.31.1 exposes no stable machine-readable surface dump (CLI has only generate/scaffolding/pipeline; print_repr emits Rust {:#?} of uniffi_meta::Metadata with no serde). Also rejected CLI + REST + WASM - the .d.ts drift gate cannot live in check-rust (only build-check-wasm runs there; wasm-bindgen-cli is installed only by the web job).
    Reverses: add a fourth artifact + generator group in reference-groups.mjs.
    Filed: the lane files "docs: SDK/FFI/WASM reference (S-Z10)" carrying the uniffi evidence and links it.

  2. How pages reach Starlight — a bun prebuild generator writing Markdown into a gitignored content path.
    Taken: scripts/gen-reference.mjs on node: builtins, invoked from package.json's dev/build scripts; zero new dependencies; pages are ordinary collection entries so Pagefind, the link validator, PageTitle and the notranslate pass apply unchanged.
    Rejected: starlight-openapi (a new Web-row dependency covering REST only; developer-docs.md:116 wants a generator over a renderer); a custom Astro content loader (replaces docsLoader for one section).
    Reverses: delete gen-reference.mjs, restore package.json scripts, add the plugin and its dependencies.md Web row.

  3. CLI emitter shape — a second [[bin]] in capsule-cli dumping clap's tree as JSON.
    Taken: gen_cli_surface.rs mirroring capsule-server/src/bin/gen_openapi.rs (default file + --check, byte comparison, trailing newline), walking clap::Command via get_subcommands/get_arguments; serde_json already a dependency.
    Rejected: clap_mangen + clap_complete (S-Z8's text names them) - neither in Cargo.lock, both need dependencies.md rows, and neither produces the JSON the docs build reads.
    Reverses: add the two crates with rows and a second emitter bin.

  4. Generated-page granularity and navigation — one page per hand-ordered group, from a table both the generator and astro.config import.
    Taken: scripts/reference-groups.mjs as the single ordered group table (CLI commands page; six REST groups: version, auth, albums, quota, moderation, well-known); the generator fails on any operation matching no group.
    Rejected: one page per operation with autogenerate - 59 pages ordered by filename, which developer-docs.md:157 forbids; the document carries no tags (59/59 untagged).
    Reverses: one file per operation and sidebar autogenerate; reference-groups.mjs keeps labels only.

  5. clap_mangen/clap_complete output is dropped from S-Z8's scope.
    Taken: S-Z8 lands the command-tree JSON and its --check, and nothing else; the slice's Deliverable sentence in SLICES.md is trimmed to match, so the row that reads done is true of the text above it.
    Rejected: Keeping man pages and shell completions in S-Z8 - neither crate appears anywhere in Cargo.lock, so both are new dependencies owing design/dependencies.md rows, and neither produces the description artifact the docs build reads. They are install artifacts, not description artifacts.
    Reverses: add the two crates with their dependencies.md rows and a second emitter binary; the JSON dump stays regardless.

  6. The S-Z8/S-Z9 status flips land in slice 4 of this pull request, not as a separate series.
    Taken: One pull request carries the pages and the record of them, so a reviewer sees the claim and its evidence together and neither can merge without the other.
    Rejected: Flipping the rows in slice 1 - it would have recorded a landing before the pages existed, which is the failure mode the status column is there to catch.
    Reverses: revert the slice-4 commit; the rows return to ready/blocked and the pages stay.

  7. Eleven REST groups, not the six the plan named.
    Taken: version, well-known, auth, upload, albums, sync, storage, shares, drops, quota, moderation - covering all 51 paths and 59 operations with no path matched by two groups. The plan's six covered 29 paths; the generator fails on an ungrouped operation, so six groups could not have built. Names track the surface map in design/api-surfaces.md so a reader arriving from a design document finds the page named after the surface they were reading about.
    Rejected: The plan's six plus a catch-all "other" group - a heading nobody chose, which is exactly what the fail-on-ungrouped rule exists to prevent.
    Reverses: edit the list in reference-groups.mjs; the sidebar and the generator both follow it.

  8. Group matching is longest-prefix, not first-match.
    Taken: groupForPath scores every prefix and takes the longest, so the table's reading order and its matching order are independent.
    Rejected: First-match in declared order - it makes adding a narrower group (/v1/auth/totp/ under /v1/auth/) silently depend on placing it above the broader one, a trap someone reordering the table for navigation reasons would spring without noticing.
    Reverses: replace the scan in groupForPath with a find.

  9. Each surface's hand-written overview is a sibling file, not an index.md inside the generated directory.
    Taken: reference/cli.md and reference/api.md (routes /reference/cli/ and /reference/api/), with reference/cli/ and reference/api/ gitignored and pruned from the docs-truth walk in their entirety. Deviates from the record's cli/index.md / api/index.md spelling, same routes.
    Rejected: reference/cli/index.md beside generated siblings - it puts a committed file inside a directory that is otherwise build output, so the ignore rule becomes a negation pattern and "is this file generated?" stops being answerable from its path.
    Reverses: move the two files into the directories and swap the ignore rules for negations.

  10. The docs-truth walk exclusion lives in capsule-docs/scripts/lib/walk.mjs, not in docs-truth.mjs. Manifest correction.
    Taken: walk.mjs gains a repo-relative SKIP_PREFIXES list beside its existing SKIP_DIRS, documented as the third instance of the rule its header already states; docs-truth.mjs (in the manifest) gains the cross-reference where it declares its scope as committed text against committed text.
    Rejected: Putting it in docs-truth.mjs as the record's manifest says - not implementable: each check calls walkFiles(root, predicate) itself, so docs-truth.mjs has no injection point. The alternatives were editing all three check files (three copies of one rule, and further outside the manifest) or leaving generated pages in the walk, which makes a check's verdict depend on whether the site happens to have been built locally.
    Reverses: delete SKIP_PREFIXES and its uses.

  11. The emitter binary prints no prose.
    Taken: gen_cli_surface writes the output path on success, is silent on --check success as cargo fmt --check is, and carries the stale-artifact message on the Result via format!. The reason is stated in the file's module doc.
    Rejected: Mirroring gen_openapi's println!("Wrote {}", …) - xtask i18n-guard scans capsule-cli/src/** for literals in print/eprint/eyre/bail, and locales/i18n-guard-allowlist.txt says in as many words not to add a CLI line to make new output pass. Also rejected: adding cli.* catalog keys for it - it would put strings no user can reach into every translation catalog.
    Reverses: narrow the guard's CLI root to exclude src/bin/, then print freely.

  12. command_tree() documents the English-bundle requirement rather than calling Bundle::for_locale("en").
    Taken: Every string in the tree is today a compile-time &'static str from a clap attribute or doc comment, so there is no bundle to pin and an unused one would be dead code. The doc comment states the property (deterministic, locale-independent, never reads cli_bundle()) and the constraint on whoever localizes help text next, and a test asserts byte-identical output across two POSIX locale environments.
    Rejected: Constructing an English bundle now - it would not be read, and a reader would take an unused binding for a guarantee.
    Reverses: resolve help text through Bundle::for_locale("en") in command_tree when a clap string becomes localizable.

  13. Links inside artifact prose are rewritten by the generator, not fixed in the annotations.
    Taken: A repo-relative path to a design document becomes the Starlight route the same file serves; a rustdoc intra-doc path, which no web server resolves, keeps its text and loses its link. Both forms are live in the committed OpenAPI document and all three instances failed starlight-links-validator.
    Rejected: Fixing the annotations in capsule-server - rule 2 would normally demand exactly that, but those links are correct for rustdoc, which is also a published surface. "Correct in the crate, wrong on the site" is a property of republishing, not an error in the source. It is also outside this lane's manifest.
    Rejected: Dropping every non-absolute link - the design-doc references have an exact site equivalent, and dropping them would lose a reference the reader can follow.
    Reverses: delete rewriteLinks and its four call sites; the three links become build failures again.

  14. Fix the self-contradictions this change created, where they sit.
    Taken: developer-docs.md's preamble ("Every surface named below is Planned or Blocked … holds no generated content today") and its "Why REST is blocked" paragraph are rewritten to the landed state, and SLICES.md's blocked-row narrative sentence drops S-Z9 and reads six. A document whose prose argues with its own table two screens down is worse than either version alone. PR [DOCS] Reset the trackers to the tree, and add a checked ROADMAP.md #422 also rewrites that preamble; a textual merge on a paragraph is accepted.
    Rejected: Leaving them — a file contradicting its own table, in the document that owns the whole pipeline.
    Rejected: Recounting SLICES.md's row-count paragraph as well — it collides with [DOCS] Reset the trackers to the tree, and add a checked ROADMAP.md #422 and roadmap: final API audit and tracker recount after the programme's pull requests #417 for no gate benefit. It is W-FINAL's (roadmap: final API audit and tracker recount after the programme's pull requests #417) by the run's ownership rule, and is owed there: after this PR the counts are 95 done / 36 ready / 6 blocked.
    Reverses: restore the three paragraphs.

  15. Multi-media-type carriers and oneOf/allOf/anyOf schemas are FATAL now, not deferred.
    Taken: readOpenApiDocument refuses both, naming the operation or the schema, and both join the module header's fatal list. The renderer shows one body per carrier and flattens a schema to a property table; neither can express a choice of media type or a union, and the silent answer in both cases is a confident lie — "this endpoint accepts only JSON" when it also accepts CBOR, or an empty model.
    Rejected: Rendering the first media type and ignoring composition — the silent case, which publishes the wrong page and fails nothing.
    Rejected: Full renderer support now — outside this lane. Nothing is filed for it: the fatal error names itself and the operation, which is a better tracker than an issue nobody reads.
    Reverses: delete assertRenderable and its call.

  16. Take the recorded reversal for decision 11: carve capsule-cli/src/bin/ out of i18n-guard's CLI root.
    Taken: One NEVER_SCANNED constant in xtask/src/i18n_guard.rs and a prefix check in scan_surface; the emitter prints plain English again. It is an audience carve-out — the distinction the module doc is built on — for tooling run from mise and CI and never installed, not a narrowing of the rule for the capsule surface. Manifest widened by that one file, recorded here. [FIX] ICU plural evaluation in the Rust formatter, and the swift-computed-property detector #420 also edits that file (the detector regex), so this hunk is kept minimal and far from it.
    Rejected: Leaving the emitter prose-free with a note — it left the guard's pattern limitation shaping the code it guards, which is the tail wagging the dog.
    Rejected: Teaching the guard to recognise developer-facing output — xtask design work outside this lane.
    Reverses: delete NEVER_SCANNED and its use, and the emitter's three lines go back to a path and an exit code.

  17. The repo-level generate assertion renders into a temp root, never the working tree.
    Taken: It seeds a temp root from the two committed artifacts (the existing fixtureRoot() shape), so the assertion is still about this repository while test-docs and build-docs stay parallel under check-docs.
    Rejected: Serialising test-docs before build-docs — it makes the gate slower for every contributor to accommodate one test.
    Reverses: pass repoRoot to generate again.

  18. reference/api.md does not reproduce the negotiation header table.
    Taken: One sentence linking design/api-surfaces.md#negotiation-across-transports for the contract, and saying that the per-operation parameters and response headers on the generated pages are what the wire carries at this commit. The table is a design commitment, not what every route implements; a reference page asserting it is wrong today and drifts again tomorrow. Lane server: the protocol headers the design puts on every route are on four operations #404 is implementing the full set — when it lands the generated pages show it and this page needs no edit, which is the point.
    Rejected: Scoping the table per route — it drifts again, and by hand.
    Reverses: paste the six rows back.

  19. Artifact prose is trusted Markdown, and that is stated where the escaping is explained.
    Taken: escapeCell's doc says the prose comes from reviewed Rust source in this repository, so emphasis, links, and inline code in it are the author's intent and pass through. The two escapes repair characters whose meaning changes when prose written for a doc comment is republished inside a table; they are not a security boundary. An unclosed < outside a code span in a doc comment is the author's bug, fixed in the doc comment.
    Rejected: HTML-sanitising artifact prose — it would strip intentional markup and imply a threat model the source does not have.
    Reverses: delete the paragraph; the behaviour is unchanged either way.

  20. The schema walk keys its cycle guard on the shallowest depth, and the bound clears the committed closure.
    Taken: seen is a name→shallowest-depth map, re-expanding when a shallower path arrives; MAX_SCHEMA_DEPTH rises from 2 to 4. Keyed on "seen at all", the answer depended on traversal order — reach a model deep first and its children are cut, and the shallower path that would have expanded them is refused. The bound mattered too: the deepest chain in the document is 3, so WireBlobRole was named on /reference/api/sync/ and defined nowhere. Both are tested, and both tests were checked against the old code. The page prose now states the rule the code implements.
    Rejected: Only the shallowest-depth fix — measured: it does not reach WireBlobRole, which is at depth 3.
    Rejected: An unbounded closure with a plain visited set — simpler, and it terminates, but it drops the stated bound that keeps a pathological document from producing an enormous appendix.
    Reverses: restore the visited set and the bound of 2.

  21. The root long_about uses Markdown list markers, which changes shipped capsule --help.
    Taken: became - in the clap annotation. The text is the root long_about in the committed command tree and is rendered as prose on the reference page, where bullet characters soft-wrap into one run-on paragraph; - renders as the list it already is, and a terminal shows a - list just as well. This is rule 2 in practice — the page was wrong, so the annotation it came from is what changed — and it was applied without being recorded, which this entry fixes.
    Rejected: Translating to - in the renderer — it would hide, in a generator, a fix that belongs in the source, and every other consumer of the annotation would keep the worse text.
    Reverses: restore and translate in describeArg.
    Note: if S-I8 (lane cli: help text from the catalogs, an enrichment read surface, and the capture-timestamp repair (S-I8, S-B18, S-B17) #413) moves this text into the catalog, the marker choice travels with it.

  22. The fatal checks match the grammar they claim to enforce.
    Taken: The setext underline is = + / - + per CommonMark, not two or more, and the composition check walks inline properties, items, and additionalProperties rather than only a named schema's root — naming the property path as well as the schema. Both defects were in the guards added the round before, and both had the same shape: the check was narrower than the rule it was written to enforce, so the input it existed to catch walked straight through. A lone - under a title-like line shipped an undemoted heading into a page body; a oneOf one level down fell through typeOf to object and would have rendered as a row calling a union a plain object. A $ref is deliberately not followed — its target is a named schema the scan reaches on its own pass, and following it would report one composition once per reference. The module header said three fatal modes and listed six.
    Rejected: Leaving both as documented-unreachable on today's document — the whole argument for making these fatal was that a silent wrong answer is worse than a build failure, and a guard that misses the input it names is exactly that silent wrong answer wearing the guard's clothes. "Unreachable today" is also what was said about the cases these checks cover.
    Reverses: restore {2,} in SETEXT_UNDERLINE and drop the recursion from assertNoComposition.
    Note: verified after the fix that a recursive scan of capsule-server/openapi.json finds no composition anywhere and that readOpenApiDocument still accepts it; both new tests were checked against the pre-fix code and fail there (2 and 3 respectively).

Unresolved review notes

  • capsule-docs/src/content/docs/design/developer-docs.md still carries two statements this
    change makes stale — the "Every surface named below is Planned or Blocked"
    sentence in its preamble and the "Why REST is blocked" paragraph. Both are left
    deliberately: PR [DOCS] Reset the trackers to the tree, and add a checked ROADMAP.md #422, on a sibling stack, rewrites them, and this lane edits only the
    surface-status table rows to keep the two changes from colliding. Forecast collision
    accepted.
  • No gate proves a generated page is semantically faithful to the artifact it came from.
    Recorded under ## Risks and rollout.
  • SLICES.md's row-count paragraph is owed a recount and does not get one here. It
    still reads 93 done / 55 done* / 37 ready / 9 part / 7 blocked / 4 post-v1; after this
    PR the true figures are 95 done / 36 ready / 6 blocked. That paragraph belongs to
    W-FINAL (roadmap: final API audit and tracker recount after the programme's pull requests #417) under the run's ownership rule, and recounting it here would collide with
    [DOCS] Reset the trackers to the tree, and add a checked ROADMAP.md #422 and roadmap: final API audit and tracker recount after the programme's pull requests #417 for no gate benefit (decision 14). The blocked-row narrative sentence,
    which this change made false by name, is corrected.
  • check-cross-links.mjs (:70, :194) calls existsSync into the generated
    directories even though the walk no longer reads them as sources.
    A published-origin
    URL or a repo-relative path naming a generated page would resolve on a machine that has
    built the site and be reported as broken on the CI runner. Not triggered today
    verified by running check-docs-truth on a tree with no generated pages: identical
    result — because the only links into those routes are site-internal root-relative ones,
    which the check hands to starlight-links-validator. SLICES.md names /reference/cli/
    in prose but does not link it. Left because the fix belongs in check-cross-links.mjs,
    outside this lane's manifest, and wants one shared constant with walk.mjs and
    reference-groups.mjs.
  • capsule-docs/scripts/lib/markdown.mjs:32's FENCE is unanchored, the same
    fence-close bug this change fixed in the generator
    : any sufficiently long fence run
    counts as a closer, so a nested ```js inside a ```sh block ends the
    block early. Pre-existing, not caused here, and outside the manifest. It affects which
    lines the docs-truth checks read as prose.
  • The <br /> this generator writes into a response Body cell is the one piece of markup
    escapeCell must never see, and that is enforced by call-site discipline rather than by
    a type. A future cell that mixes generator markup with artifact prose would need the two
    kept apart explicitly.
  • Generated pages carry status: stable while the hand-written overviews beside them carry
    status: draft, so the site shows machine output as reviewed and human prose as not. The
    content schema offers only those two values. stable is deliberate — correctness is
    mechanical rather than review-pending — but a third generated state would say it
    better, and that is a content.config.ts change outside this manifest.

The documentation build installs bun and nothing else, so it cannot ask cargo
what `capsule --help` says. Give it a committed artifact to read instead, and a
drift gate that makes a stale one fail CI (slice `S-Z8`).

`capsule_cli::cli::command_tree()` walks the clap tree built from compile-time
attributes and returns JSON. It is the crate's only new public surface: `Cli`
stays `pub(crate)` because the parsed command is dispatch state, not API.

The output is deterministic and independent of the process locale, both because
the artifact is byte-compared by its own gate. Subcommands are sorted by name so
reordering an enum variant cannot churn the file; arguments keep declaration
order, which for a positional is its position. `Command::build` is not called,
so clap's synthesized `--help` is not described sixteen times over, and a
boolean flag is not documented as taking `true` or `false`.

`gen_cli_surface` mirrors `gen_openapi` argument for argument — `[FILE]`
default, `--check`, byte comparison, trailing newline — so the two description
artifacts are one thing to remember rather than two. It adds no dependency:
serde_json and clap were already here.

`cli-surface-check` joins `check-rust` beside `openapi-check-kynos`.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 2, 2026

Copy link
Copy Markdown

Deploying capsule with  Cloudflare Pages  Cloudflare Pages

Latest commit: 00f8dce
Status: ✅  Deploy successful!
Preview URL: https://3c8358ac.capsule-22k.pages.dev
Branch Preview URL: https://docs-reference-generation-41.capsule-22k.pages.dev

View logs

…d tree

`/reference/` held one page saying nothing was published there yet. It now
publishes the command line, generated from `capsule-cli/cli-surface.json` by a
bun prebuild step (slice `S-Z8`).

`scripts/gen-reference.mjs` runs on `node:` builtins alone and adds no
dependency, which is what lets it run in the docs job as it stands — bun and
nothing else, no cargo. It writes ordinary content-collection entries, so
Pagefind indexes them, the link validator checks their anchors, the `PageTitle`
override renders their badge, and the notranslate pass marks up their terms. An
embedded renderer that mounted its own application would have forfeited all
four.

The pages are gitignored: a committed copy of generated output is a second
source of truth that can disagree with the artifact it came from, and rule 2 of
`design/developer-docs.md` exists so it cannot. Hand-written prose stays in one
overview per surface, `reference/cli.md` beside the generated directory.

`scripts/reference-groups.mjs` is the ordered page table. `astro.config.mjs`
builds the `Reference` sidebar from it and the generator decides which pages
exist from it, so the sidebar stays hand-curated as `developer-docs.md`
requires while a page with no navigation entry stops being expressible.

A missing, unparseable, or unknown-schema artifact fails the build naming the
path. It never emits a stub: an empty reference page is the confidently-wrong
case the design doc puts above a missing one.

Headings inside artifact prose are demoted rather than interpolated — an
operation description opening at `#` would otherwise inject a second h1 into a
page whose h1 is the Starlight title.

The `docs` path filter now names every artifact the build reads, not just the
site, so a change to a described surface cannot publish a stale page. The
docs-truth walk skips the generated directories for the mirror-image reason
`rawshift/` is skipped: they exist on a machine that has built the site and on
no CI runner, and a check that read them would answer differently in the two
places.
… keys once

Locale independence is the property the drift gate rests on — the artifact is
byte-compared, so a string negotiated from the environment would make
`cli-surface-check` pass or fail according to the developer's `LANG` — and
nothing asserted it. Render the tree under `en_US`, `tr_TR`, and `ja_JP` and
compare; `tr-TR` because it is the locale that breaks case folding, and
`LC_ALL` because that is what `cli_bundle` reads first.

Cover both branches of the `long_about`/`long_help` dedup, which decides whether
the artifact carries the same paragraph twice, over a synthetic `Command` — the
real surface has no argument with a distinct long help, so only a fixture
reaches the branch that keeps one.

The document's field names move into one `field` block. The shape is a
projection of clap's builder API and no type here has it, so it stays
hand-built; naming the keys once is what keeps that from meaning spelled ad hoc,
since a typo is a field `gen-reference.mjs` silently never finds.

Record `ArgAction::Count` as unreachable on today's surface, and why it is
matched anyway.
…ment

`/reference/api/` now publishes all 51 paths and 59 operations of the committed
Kynos document, across eleven hand-ordered group pages, plus a hand-written
overview carrying what is true of every endpoint — the auth model, the
negotiation headers, and the error contract — so no generated page repeats it
fifty-nine times (slice `S-Z9`).

Grouping is by hand because the document offers nothing to group by: none of
its 59 operations carries a tag. Autogenerating would have meant 59 pages
ordered by filename, which `design/developer-docs.md` forbids, and would have
scattered the four operations of the upload protocol across the alphabet. The
group names track the surface map in `design/api-surfaces.md`. Matching is
longest-prefix, so the table's reading order and its matching order stay
independent: adding a narrower group later cannot silently depend on where it
sits.

An operation no group claims fails the build, naming it.

Links inside artifact prose are rewritten for the site. A repo-relative path to
a design document becomes the Starlight route the same file serves; a rustdoc
intra-doc path, which no web server resolves, keeps its text and loses its link.
Both forms are live in the committed document, and both are correct where they
were written — this is republishing, not an error in the source.

Also fixes defects found reviewing the CLI half against the real documents:

- Generated output is cleared before it is rewritten. A page a later run no
  longer emits used to stay on disk, and the directory is gitignored, so
  nothing showed it: Astro kept routing and indexing a page no artifact
  described, on one machine and on no CI runner.
- A nullable type renders as `string \| null`. Unescaped, the separator opened a
  fourth column and shifted every cell in the row.
- A closing fence must carry no info string, or a nested ```js inside a ```sh
  example ends the block early — demoting the example's comments and skipping
  every real heading after it.
- `<` is escaped: Markdown passes raw HTML through, so an angle-bracketed
  placeholder vanished from the page.
- A required option is spelled in the usage line instead of folded into
  `[OPTIONS]`, which was handing the reader a command that fails to parse.
- Blank-line tidying skips fenced blocks, so the generator stops editing the
  examples it quotes.
- The artifact is rejected when it parses but describes nothing, rather than
  publishing an empty heading under a stable badge.
- Frontmatter titles are quoted, `preview`/`deploy` build first, and the `docs`
  filter names the root `biome.jsonc` that `capsule-docs` extends.

The root `long_about` uses Markdown list markers so the generated page renders
the list it already is — rule 2 in practice: the page was wrong, so the
annotation it came from was what changed.
The surface table in `developer-docs.md` said the CLI was Planned and REST was
blocked on rendering. Both now publish, so both rows read Landed, naming the
emitter and the gate that keeps each artifact current.

`S-Z8` and `S-Z9` move to `done` with a verified line each. `S-Z8`'s Deliverable
drops man pages and shell completions: neither `clap_mangen` nor `clap_complete`
is in `Cargo.lock`, both would owe a `dependencies.md` row, and neither produces
the description artifact the docs build reads — they are install artifacts, and
they belong to a packaging slice.

`S-Z10` stays `ready` and carries the evidence for why, so the next attempt does
not rediscover it: uniffi 0.31.1 exposes no stable machine-readable surface dump,
and the wasm `.d.ts` gate cannot run where `check-rust` runs. It is filed as its
own issue.
… mislead

Review repairs on the two reference surfaces, all of the same shape: where the
generator's silent answer would be a confident lie, it now stops and names what
it cannot render.

Fatal, added to the module header's list and enforced in `readOpenApiDocument`:

- A request or response carrier offering more than one media type. The renderer
  shows one body per carrier, so a second was dropped silently and the page
  claimed an endpoint accepting JSON and CBOR accepted only JSON.
- A schema composing with `oneOf`, `allOf`, or `anyOf`. A property table cannot
  express a union or an intersection, and rendered one as an empty model.
- A setext heading in artifact prose. Leaving it undemoted put an h1 in the page
  body — the exact defect demotion exists to prevent — and published looking
  fine. ATX in the doc comment is the fix, and now the build says so.

None is reachable on the committed document; each is how the first one to appear
gets handled instead of shipped.

Also:

- Table cells are escaped once, over the assembled cell, rather than fragment by
  fragment. Escaping only the help text left `Values:` and `Default:` raw, so a
  default of `a|b` opened a column of its own — the defect already fixed for
  types, reintroduced one layer down.
- The schema-appendix walk keys its cycle guard on the shallowest depth a model
  was reached at, not on having seen it. Keyed on the latter, the answer depended
  on traversal order: reach a model deep first and its children are cut, and the
  shallower path that would have expanded them is refused as already-seen.
- The `$ref` bound rises to 4, above the committed document's deepest chain of 3.
  `WireBlobRole` was named on `/reference/api/sync/` and defined nowhere; the
  whole closure costs one further model across all eleven pages. The page prose
  now states the rule the code implements.
- Schema-level descriptions go through `rewriteLinks` like every other prose
  site. A model's doc comment cites design documents as freely as a handler's.
- `escapeCell` records that artifact prose is trusted Markdown: it comes from
  reviewed Rust source, so emphasis and links are the author's intent. The two
  escapes repair characters whose meaning changes inside a table; they are not a
  sanitizer, and an unclosed `<` outside a code span is the author's bug.
- The repo-level test renders into a temp root seeded from the committed
  artifacts. Generating into the working tree raced the Astro build reading it,
  since `check-docs` runs `test-docs` and `build-docs` in parallel and `generate`
  clears its output first.
- The two-locale test puts `LC_ALL`/`LANG` back.

`gen_cli_surface` prints plain English again. `xtask i18n-guard` gains a
`NEVER_SCANNED` carve-out for `capsule-cli/src/bin/`, which is build tooling run
from mise and CI and never installed — an audience carve-out, the distinction
that module is built on, not a narrowing of the rule for the `capsule` surface.
Removing the carve-out catches all three of the binary's lines, so it is doing
the work.
`developer-docs.md` still opened by saying every surface below was Planned or
Blocked and that `reference/` held no generated content, and still explained why
REST was blocked — three paragraphs contradicting the table two screens down
that this branch had already flipped to Landed. They now describe the landed
state and record how the block cleared, so the document does not argue with
itself.

`SLICES.md`'s blocked-row narrative counted `S-Z9` among the rows waiting on a
decision. Moving that row to `done` made the sentence false; it now reads six and
does not name `S-Z9`. The row-count paragraph is deliberately untouched — it is
another lane's to recount.

Two overstatements in the new prose, both found by reading it against the
artifacts rather than against intent:

- `reference/api.md` reproduced the six-row negotiation header table from
  `api-surfaces.md`. That table is a design commitment, not what every route
  implements today, and a reference page asserting it would be wrong now and
  drift again later. It links the contract and says that the parameters and
  response headers on the generated pages are what the wire carries at this
  commit — so when the full set lands, the generated pages show it and this page
  needs no edit.
- `reference/cli.md` claimed every command that opens a library accepts
  `--passphrase-stdin`. Three do — `import`, `push`, `cull` — and the artifact
  says so. The `capsule library` subcommands are not among them because they
  read the version file, the sidecars, and the index, none of which is sealed.
Both defects were in the guards added last round, and both had the same shape:
the check was narrower than the rule it was written to enforce, so the input it
existed to catch walked through it.

CommonMark's setext underline is `= +` or `- +`, not two or more. Requiring two
meant `Title` over a lone `-` was not recognised, and shipped an undemoted
heading into a page body — the exact defect the check exists to prevent,
arriving through the check against it. The fence, table, list, and thematic-break
exclusions are unchanged, and are covered.

The composition check read `oneOf`/`allOf`/`anyOf` at the root of a named schema
only. A composition one level down, in `properties.<name>` or in `items`, fell
through `typeOf` to `object` and would have rendered as a row calling the field
a plain object — a confident lie about a union, which is what the check is for.
It now walks inline `properties`, `items`, and `additionalProperties`, and names
the property path as well as the schema. A `$ref` is not followed: its target is
a named schema this scan reaches on its own pass, and following it would report
one composition once per reference.

Neither reaches the committed document: a recursive scan of
`capsule-server/openapi.json` finds no composition anywhere, and
`readOpenApiDocument` still accepts it.

The module header said three fatal modes and listed six.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant