Docs sweep: resync the cross-cutting maps and the stale as-built claims - #423
Merged
Conversation
The per-PR spec updates have held up well. What drifted is the layer above them: the maps that claim to list everything. This resyncs them. Doc maps - docs/README.md was missing CAPABILITIES.md. CLAUDE.md calls that a bug, so the map now also states the rule it broke: a doc in specs/ that is not in the Orientation list is a bug, fixed in the same change. - docs/README.md and docs/dev/contributing.md both pointed readers at the "Documents" section of CLAUDE.md. That section no longer exists — CLAUDE.md was rewritten. Both now point at docs/README.md # Specs, which is the list they were describing. - docs/progress/README.md was missing two entries, cloud-vm-boot-proof.md (C2) and cloud-seed-delivery.md (the C3a cloud-lane half). The index is the "where we are now" view, so a missing row is a hole in it. 191 rows -> 193, matching the 193 entry files. One NEXT.md, not two Two lived side by side: docs/specs/NEXT.md (tiered, 521 lines) and a root NEXT.md (37 lines) that different PRs edited without noticing the other. docs/specs/NEXT.md is canonical — every spec links to it as a sibling. The root file is folded in and removed: - Box <-> cloud authentication merged into the existing Tier 1 entry, which did not yet record where #408 left it (a bare box_id on a public endpoint, accepted only because the ask is a read) or that the seed is the write-once delivery channel for the real credential. - Encrypt hosted enrollment credentials at rest -> Tier 3. - Per-app disk quota for hosted tenants -> Tier 3. - The daemon-free registry sizer -> Tier 4, author-surface group. - Dropped: admin-port isolation (shape locked, tracked in #187) and the acme-dns public face (marked DEPLOYED, remaining work is cloud-side). Both fail the doc's own rule that a locked or resolved topic leaves it. CLAUDE.md now names docs/specs/NEXT.md and docs/specs/DECISIONS.md by path and says there is no root copy of either. As-built claims that had gone stale - architecture.md said "no LUKS-unlock flow, no malmo-storage-ready.target". Both exist: the units ship in dist/systemd/, the reporter is cmd/malmo-storage-verify, and LUKS + first-boot TPM enrollment is proven in the QEMU medium lane. The real gap is the data-drive half — no mergerfs, no /srv/malmo pool, no UI to add or unlock a drive. README claimed the opposite of architecture.md here; they now agree. - architecture.md # Inside the brain presents itself as the whole of internal/ but was missing five packages: health, notify, applog, systemlive and storageverify. Added, with storageverify marked as not a brain package. - CLAUDE.md said of host-agent-real "PAM verify is real; the rest is still being built". Its own architecture.md row lists ten-odd real reporters plus discovery, brain launch and the update path. - README's layout table listed catalog/, deleted at the thin-client cutover (#420). Dropped; api/ and dist/ added; the cmd/ and internal/ lists brought up to date (both were also stale in CLAUDE.md). - README listed "the signed remote app-store fetch" as not built. It shipped, and DECISIONS.md 2026-07-02 removed the signature deliberately. Replaced with the actual state of the update streams. - README's quickstart says to install Whoami from the catalog without saying the catalog is no longer in this repo. Added the pointer to make dev-app. VERSION: dev said 0.6.0, main said 0.10.0 Releases bump VERSION in the dev->main PR, so v0.7.0 through v0.10.0 landed on main and never came back. Every dev build stamped itself 0.6.0 and Settings -> About reported a version four releases old. Set to 0.10.0 to match main — not 0.11.0, because VERSION is the release trigger and picking the next number is the maintainer's call. Also picks up #418 (two gaps from the laminar import), which merged to main and never reached dev. Claude-Session: https://claude.ai/code/session_012y33wKNxu7iUd1wLa95TXg
Round 1 fixed the maps. This checks the specs themselves against what the code does, focusing on areas the last ~30 PRs touched. architecture.md # What is not built yet had drifted furthest That section is the one readers trust for "what is real", and four of its claims were false: - "Managed-DB-in-production stays gated on a provisioning re-architecture off docker exec". That re-architecture landed in #185 on 2026-06-15 and DECISIONS.md 2026-06-15 flips the 2026-06-14 gate it cites. Provisioning runs in a one-shot --rm client container, EXEC stays denied. - "the brain doesn't surface health issues, the bell doesn't exist, no chrony integration". All three exist: internal/health raises typed issues onto GET /api/v1/health and HealthBanner, internal/notify + NotificationBell.vue are the notification centre with per-category mute, and clockhealth parses real chronyc tracking. Telemetry is the part that is still true — consent is recorded, no client sends anything — so the bullet is now about telemetry, with off-box notification transports split out as its own gap. - "Login.vue is kept in the tree but not routed (single-user dev phase)". It is rendered: App.vue picks between Setup, Login and the dashboard from auth state. What is absent is a /login *route*, which is a different claim. - Storage said "no LUKS-unlock flow, no malmo-storage-ready.target" — corrected in round 1, restated here for the record. Also: the web-ui row said shadcn components were "not yet copied in", but components/ui/ holds two deliberately hand-written ones (#261). A host-protocol endpoint that was never written down POST /v1/system/set-timezone is implemented end to end — hostclient, protocol.SetTimezoneRequest, a real timedatectl reporter, wired in both build profiles — and BRAIN_HOST_PROTOCOL.md, which CLAUDE.md calls the source of truth for the host protocol, had no entry for it. Only a passing "misc host state — time zone" in the scope list. Documented with its wire shape, who validates the zone (the brain, not host-agent), and the note that clock *sync* is a separate read-only axis. Three env vars the code reads and no spec named - MALMO_UPDATE_BRAIN_REPO / MALMO_UPDATE_UI_REPO. UPDATES.md # 8.4 says an answer is refused if it "points at an unexpected repository" without ever saying what is expected (ghcr.io/malmoos/brain and .../ui) or that both are overridable. The CI boot proof depends on that override. - MALMO_RELEASE_BASE_URL, documented in RELEASE_MANIFEST.md # Failure modes, with why an override is safe: the base URL is not the trust boundary, the signature is. The ghcr public flip already happened BUILD.md # 6 said the two control-plane packages were still private, so "no box can pull them anonymously" and the published-publicly decision was "not yet in effect". Checked against the registry: both ghcr.io/malmoos/brain and ghcr.io/malmoos/ui answer an anonymous pull and carry latest plus every tag from v0.6.0 to v0.10.0. This one mattered — a box's update target names those repos by default, so the stale text implied the hosted update path could not work on a real box. Publishing no longer means "upload to the provider" #352 removed the provider-snapshot upload, and BUILD.md and contributing.md say so. CLAUDE.md and hosted-boot-proof.md still described publish=true as uploading the image to a hosting provider. It attaches the compressed image to the GitHub Release and pushes the two ghcr images; the lane holds no provider credential. Smaller - The workflow's own comment said "three QEMU boots". It is five on a push or dispatch and one on a PR, both set right below it in MALMO_CLOUD_BOOTS. - CLAUDE.md's standard-field list gained `name` and `uid`, and now records that `retry_after` (auth.go, a duration string) and `retry_after_s` (ratelimit.go, an int) are two spellings of one fact. Both are documented because both are in the code; collapsing them is a code change, not a docs one. Claude-Session: https://claude.ai/code/session_012y33wKNxu7iUd1wLa95TXg
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| docs/architecture.md | Updates the as-built component and route inventories; the previously reported route omissions are fixed. |
| docs/specs/NEXT.md | Consolidates open design topics and now accurately separates implemented update-target reads from authentication-blocked writes. |
| docs/specs/BRAIN_HOST_PROTOCOL.md | Documents the existing set-timezone protocol operation. |
| docs/specs/STORAGE.md | Aligns documented persistent storage paths with the current on-disk layout. |
| docs/README.md | Resynchronizes the specification index and repairs stale navigation. |
| VERSION | Resynchronizes the development branch version with the current release version. |
Reviews (5): Last reviewed commit: "Fix two P2 findings from the Greptile re..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Audits the specs I had scoped out of round 2 — the ones untouched since June.
Most held up (CONTROL_PLANE.md and SERVICE_PROVISIONING.md were both current).
What did not is the on-disk layout, and it was wrong in a way that matters.
The brain's database is not where three specs said it was
Code opens `filepath.Join(stateDir, "malmo.db")`, and host-agent sets
`StateDir = /var/lib/malmo/state`, so the real path is
`/var/lib/malmo/state/malmo.db`. The specs carried two different wrong answers:
- `/var/lib/malmo/brain/state.db` — STORAGE.md, THREAT_MODEL.md,
USERS_AND_GROUPS.md
- `/var/lib/malmo-state/brain.db` — LOCAL_ANALYTICS.md
USERS_AND_GROUPS.md is the one that stings: it is a **recovery instruction**
("stop the brain, replace <path> from the most recent snapshot"). Anyone
following it would have restored to a path the brain never reads, and seen no
error. It now also says to bring the `-wal` sidecar, because the brain opens
WAL and restoring the main file alone yields a valid *older* database — the
same reason `internal/hostagent/cpupdate` copies the sidecars around a
control-plane update.
App instances and managed services are nested one level deeper
Everything the brain writes lives under the one state dir: instances are at
`state/instances/<id>/` and managed-service data at
`state/services/<kind>-<version>/`. The specs said `/var/lib/malmo/instances/`
across eight files, and STORAGE.md invented a `managed-services/` sibling that
does not exist. Corrected everywhere except DECISIONS.md, which is a historical
log and stays as written.
STORAGE.md's mount-layout block is the doc that owns this, so it now shows the
real tree and states the two consequences that are silent when got wrong: a
restore targets `state/malmo.db` with its WAL, and disk-usage accounting walks
`state/instances/<id>/`.
Worth flagging: this is the code diverging from STORAGE.md's intended layout at
some point, not the docs being sloppy. I moved the docs to match reality
because reality is what an operator meets. If the intent was the flatter layout
the specs described, that is a migration and a maintainer call.
A state root that never existed
TELEMETRY.md and BOOT.md both wrote paths under `/var/lib/malmo-state/`, a root
nothing ever created. BOOT.md's `.bootstrapped` file is superseded too: the
brain records `first_run_complete` in `box_meta`, and
`malmo-prepare-wizard.service` does not exist in `dist/systemd/`. Kept the
design text, added an as-built note, so a later implementer reads the brain's
state instead of creating a second root beside it.
Also
- LOGGING.md checkpointed the journald cursor in `brain_meta`. There is no such
table; the key-value store is `box_meta`.
- BRAIN_UI_PROTOCOL.md said the brain resolves icons and screenshots "inside the
app's catalog directory". After #420 there is no catalog directory — assets
are fetched from the control plane and cached with a 24h expiry.
- architecture.md's dev-tree still listed the deleted repo-root `catalog/`
(round 1 caught the copy in README, missed this one), and omitted `services/`
and the catalog asset cache.
- architecture.md never listed the **file manager** as unbuilt. `/files` is a
routed top-level destination backed by a "coming soon" stub, with no
`/api/v1/files/*` handlers and no host-agent ops. FilesView.vue's own comment
claimed FILES.md was "not yet written" — it has been since June.
`make build` and `make check-web` green.
Claude-Session: https://claude.ai/code/session_012y33wKNxu7iUd1wLa95TXg
Round 3 moved the specs onto the shipped paths but never recorded the call, so the next person to notice the mismatch has nothing to read and may "fix" the code back. DECISIONS.md gets the entry: the layout stands, the specs follow it, no migration. The reason is that /var/lib/malmo/ holds two owners' files — control-plane/ and seed.json are host-agent's, everything under state/ is the brain's — and state/ is the line between them. Flattening it would either put host-agent's files inside the brain's state dir or force the brain to know two roots. One directory is also what lets cpupdate snapshot and restore the brain in one copy, and what keeps the container bind to a single mount. STORAGE.md # mount layout carries the same reasoning inline, next to the tree it explains, plus the two failure modes that are silent when got wrong: a restore without the -wal sidecar opens fine and is quietly missing commits, and a du against the old instances/ path returns nothing rather than erroring. Claude-Session: https://claude.ai/code/session_012y33wKNxu7iUd1wLa95TXg
Round 2 added a line to CLAUDE.md calling retry_after and retry_after_s "two spellings of one fact" that were "worth collapsing to one". Re-checked against AUTH.md # Rate limiting and that is wrong. The two are separate throttles the spec keeps apart on purpose: "The two don't overlap: /login keeps its stricter per-username backoff here." The 429 + Retry-After contract is scoped to the general limiter, so the login path sending no such header is correct, and ratelimit.go's retry_after_s is consistent with its own JSON field and the header it sets. Both names are right where they are. The field list keeps both keys, now with what each one means and why a filter wanting every throttled request has to ask for both. Claude-Session: https://claude.ai/code/session_012y33wKNxu7iUd1wLa95TXg
CLAUDE.md # Working style asks for CEFR B1 in everything, including docs and code comments: short sentences, common words, one idea per sentence, no clause-stacking or em-dash pile-ups. The text I added in the earlier commits broke that rule throughout. This rewrites it. Example, from STORAGE.md. Before: Everything the brain writes sits under one state dir it is handed as MALMO_STATE_DIR (/var/lib/malmo/state on a real box; .dev/state under make dev) - the database, the per-instance dirs and the managed-service data alike. After: The brain writes everything under one folder. It gets that folder as MALMO_STATE_DIR. On a real box it is /var/lib/malmo/state. Under make dev it is .dev/state. Same facts, four short sentences instead of one long one. Rewritten: the STORAGE.md mount-layout notes, the DECISIONS.md 2026-08-26 entry, the set-timezone section in BRAIN_HOST_PROTOCOL.md, the expected-repository bullet in UPDATES.md, the base-URL note in RELEASE_MANIFEST.md, the ghcr note in BUILD.md, the proxied-assets note in BRAIN_UI_PROTOCOL.md, the recovery step in USERS_AND_GROUPS.md, the as-built note in BOOT.md, four bullets in architecture.md, five entries in NEXT.md, four passages in CLAUDE.md, and the notes in README.md, docs/README.md and FilesView.vue. No facts changed. Only the wording. make check-web green; all relative doc links still resolve. Claude-Session: https://claude.ai/code/session_012y33wKNxu7iUd1wLa95TXg
Both were real. 1. architecture.md listed only three host-agent route groups as "today the routes are", while the brain calls many more. Adding the time-zone setter to the component row made that inventory visibly wrong. It now lists every route internal/hostclient calls, checked against the package. While writing it I listed /v1/discovery/state as well. That one is wrong: host-agent serves it, but the brain never calls it, so it does not belong in a list of what the brain calls. Removed before commit. 2. NEXT.md's "Why Tier 1" for box-to-cloud auth still said the missing credential blocks building the update trigger. #402 and #408 built that trigger on a bare box_id, so the paragraph above it and this line disagreed. The justification now says what is actually blocked: the write half, meaning the box reporting its outcome (UPDATES.md # 8.4 step 5) and the fleet auto-halt that reads those reports (# 8.5). Claude-Session: https://claude.ai/code/session_012y33wKNxu7iUd1wLa95TXg
onel
added a commit
that referenced
this pull request
Aug 26, 2026
Both branches appended to the end of docs/dev/catalog-import-gaps.md, so they conflicted. #423 brought the two laminar entries onto dev (they had landed on main via #418 and never reached dev); this branch adds the penpot one. Kept both. The file says "Newest entries at the bottom", so they are ordered by date: laminar (2026-08-14), then penpot (2026-08-26). The conflict presented them the other way round. Nothing else in the merge needed a decision. The root NEXT.md deletion comes from #423, which folded it into docs/specs/NEXT.md. Claude-Session: https://claude.ai/code/session_012y33wKNxu7iUd1wLa95TXg
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The specs each PR touches are in good shape. What went stale is the layer above them: the docs that claim to list everything, and
architecture.md# What is not built yet.No behaviour changes. Docs,
VERSION, and one stale comment in a workflow file.Two findings worth reading first:
BUILD.mdsaid the ghcr images cannot be pulled without a login. They can. Bothghcr.io/malmoos/brainandghcr.io/malmoos/uianswer an anonymous pull, withlatestand every tag from v0.6.0 to v0.10.0. This mattered: a box's update target names those repos by default, so the old text implied the hosted update path could not work on a real box.mainand never reacheddev. Cherry-picked here. Worth checking if other direct-to-main merges did the same.The doc maps
docs/README.mdCAPABILITIES.md.CLAUDE.mdcalls that a bugdocs/README.md,docs/dev/contributing.mdCLAUDE.mdthat no longer existsdocs/progress/README.mdREADME.md,CLAUDE.mdcatalog/, and missedmalmo-network-verify,api/,dist/, sixinternal/packagesdocs/architecture.mdhealth,notify,applog,systemlive,storageverifyOne NEXT.md, not two
There were two: a root
NEXT.mdanddocs/specs/NEXT.md. Different PRs edited each without noticing the other.docs/specs/NEXT.mdis the real one, since every spec links to it as a sibling. The root file is folded in and deleted.Of its six topics: one merged into the existing Tier 1 entry, three moved to Tier 3 and Tier 4, and two were dropped.
The two dropped were #187 admin-port isolation and the acme-dns face. I checked both before deleting. Issue #187 holds far more detail than the entry did, and
CONTROL_PLANE.mdalready prescribes the fix. The acme-dns text is already inENVIRONMENT.md:112, including the date it went live. Nothing was lost.Stale as-built claims
architecture.md# What is not built yet is the section people trust for "what is real". Four claims were false:docker exec. Managed-DB provisioning offdocker exec(proxy-compatible) #185 did that in June, andDECISIONS.md2026-06-15 lifts the gate it cites.Login.vue"not routed" — it renders.App.vuepicks it from auth state. What is missing is a/loginroute, which is a different claim.malmo-storage-ready.target. Both units ship indist/systemd/, and LUKS with TPM enrolment is proven in the QEMU lane.Wire and config that no doc named
POST /v1/system/set-timezoneis built end to end.BRAIN_HOST_PROTOCOL.md, whichCLAUDE.mdcalls the source of truth for that wire, had no entry for it.MALMO_UPDATE_BRAIN_REPO/MALMO_UPDATE_UI_REPOdecide which repositories an update answer may name.UPDATES.md# 8.4 never said what they were.MALMO_RELEASE_BASE_URLwas unnamed inRELEASE_MANIFEST.md.On-disk paths (round 3)
The brain saves its database at
/var/lib/malmo/state/malmo.db. The specs gave two different wrong answers:STORAGE.md,THREAT_MODEL.md,USERS_AND_GROUPS.md/var/lib/malmo/brain/state.dbLOCAL_ANALYTICS.md/var/lib/malmo-state/brain.dbUSERS_AND_GROUPS.mdis the one that hurts. It is a recovery step: stop the brain, restore the database from a snapshot. Follow it and you restore to a path the brain never reads, with no error. It now names the right path, and says to bring the-walfile too.Same class of error for app data. Real is
state/instances/<id>/andstate/services/<kind>-<version>/. Eight files said/var/lib/malmo/instances/.STORAGE.mdalso had amanaged-services/folder that does not exist, andTELEMETRY.mdandBOOT.mdwrote into a/var/lib/malmo-state/root that was never created.This is the code diverging from the spec, not sloppy writing. I moved the docs to match the code, because that is what an operator meets.
DECISIONS.md2026-08-26 records why:/var/lib/malmo/holds files from two owners, andstate/is the line between them. Owner call if you disagree — I can revert that half.Spec(s) touched
NEXT.md(root file folded in and deleted),BRAIN_HOST_PROTOCOL.md,UPDATES.md,RELEASE_MANIFEST.md,BUILD.md,BRAIN_UI_PROTOCOL.md,STORAGE.md,THREAT_MODEL.md,USERS_AND_GROUPS.md,LOCAL_ANALYTICS.md,TELEMETRY.md,BOOT.md,LOGGING.md,APP_LIFECYCLE.md,APP_MANIFEST.md,APP_ISOLATION.md,FILES.md, plusDECISIONS.md,architecture.md,docs/README.md,docs/progress/README.md,docs/dev/contributing.md,docs/dev/hosted-boot-proof.md,README.md,CLAUDE.md.One new
DECISIONS.mdentry (2026-08-26), for the path call above. Nothing else flipped a locked decision.What was tested
There is no behaviour to exercise, so these are mechanical checks:
docs/specs/now appears indocs/README.md. Every progress entry has a row: 193 of 193. Both failed before..mdlink indocs/,README.mdandCLAUDE.mdresolves. Scripted, zero broken.NEXT.mdin the tree, and nothing links to the deleted path.GET https://ghcr.io/v2/malmoos/brain/tags/listreturns tags.make buildgreen. The version stamp reads0.10.0, matchingmain.make check-webgreen, for the one comment change inFilesView.vue.ci-cloud-image.ymlstill parses as YAML.Known gaps & deviations
VERSIONset to 0.10.0 to matchmain, not 0.11.0. The bump is the release trigger, so picking the next number is the maintainer's call. This makes the next dev→main merge a clean no-op. It does not fix the cause: releases bumpVERSIONin the dev→main PR and it never returns to dev, so this drifts again after v0.11.0.SPEC.md,AUTH.md,DISCOVERY.md,MALMO_NETWORK.md,TIME.md,HEALTH.md,NOTIFICATIONS.md,FIRST_RUN.md,SETTINGS.md,DASHBOARD.md,WEB_UI.md. I make no claim about those.retry_afterandretry_after_sas one fact spelled two ways, and wrote that intoCLAUDE.md. It is wrong:AUTH.md# Rate limiting keeps the two throttles apart on purpose, and each name matches its own subsystem. The note now says so. Worth knowing that claim existed in an earlier commit here.CLAUDE.mdthose are for implementation slices. This is documentation, with no behaviour change.Platform gaps
None. Not a catalog PR.
Definition of done
make buildgreen, version stamp checked.*_test.go.DECISIONS.mdentry added; no locked decision flipped.log/slogonly, plain English perCLAUDE.md.dev, PR intodevwithCloses.Closes #422