Skip to content

feat(kb): hygiene release v0.6.4 — slug data loss, audit-kb silent failure, format contract - #159

Open
George-iam wants to merge 3 commits into
mainfrom
feat/kb-hygiene-20260818
Open

feat(kb): hygiene release v0.6.4 — slug data loss, audit-kb silent failure, format contract#159
George-iam wants to merge 3 commits into
mainfrom
feat/kb-hygiene-20260818

Conversation

@George-iam

Copy link
Copy Markdown
Contributor

Driven by a full manual compaction of a production knowledge base (282 memories / 115 decisions, ~192k tokens at session start). That pass surfaced one data-loss bug, one silent-failure bug, and a format contract that was documented everywhere and enforced nowhere.

Bugs fixed

Silent data loss on non-Latin titles. toSlug/toMemorySlug were text.toLowerCase().replace(/[^a-z0-9]+/g, "-"), mapping any fully non-Latin title to "". The memory landed as bare .md — a dotfile, invisible to ls and to every shell glob over memory/*/*.md — and the next such title overwrote it. Four affected files were found on one project, two with confirmed overwrites. Slugs now transliterate Cyrillic/Greek, strip Latin diacritics, hash-fallback when nothing survives, and prefix digit-only slugs (16-07memory-16-07). saveMemory additionally refuses to overwrite a file whose stored title differs from the incoming one; same title still overwrites, which is how an entry is revised.

audit-kb could exit 0 having written nothing. A real run analysed 109 decisions correctly for four minutes, reached a conclusion, wrote zero bytes and reported success — the counters were resultText.match(/supersed/gi).length, i.e. word-counting the agent's prose. It now snapshots every entry's loaded-layer size before and after and reports the measured diff. A zero-change pass prints NO CHANGES WRITTEN, says that is a failed pass rather than a clean base, and exits 2.

Empty first page of paginated output. paginateSections never split an oversized section, so ["## Project Memories", <60KB block>] rendered page 1 as the heading alone.

Leaked tool-call markup persisted verbatim — three records ended with </description><parameter name="keywords">[...]. Storage now strips from the first stray frame tag onward and reports which fields it cleaned.

Frontmatter rewrites could delete the following field. The key-replacement regex used \s*, which matches newlines, so rewriting an empty slug: consumed the type: line under it. Caught by the smoke test on a synthetic base, not by a unit test — regression coverage added.

The root cause of the bloat

The CLAUDE.md template said "Error pattern or successful approach discovered -> call axme_save_memory immediately" with nothing anywhere saying what not to save. A month of that produced 110 research diaries and 18 session handoffs in one base's memory. The template, Cursor rules, server instructions and tool descriptions now carry the same selection test — would this help an agent a month from now who was not part of this investigation? — with an explicit negative list. The negative list is the operative half.

The format contract

The two-level scheme was already right and simply never explained: description is loaded into every session, body renders as ## Details, is not loaded, and comes back in full from axme_get_memory. Measured usage before this change: ## Details non-empty in 11 memories out of 126 — 91% put everything in the paid layer. Decisions, whose reasoning/## Reasoning pairing was explained, sat at 86 of 87.

Consequence, measured on that base: search mode cost ~65k and delivered 39-45% of the content — the worst of both modes. Writing to the catalog budget makes the catalog complete, at which point the two modes carry the same content for the same price.

So this PR states the contract in the schema field descriptions, the tool descriptions, the CLAUDE.md template and the server instructions; makes the budget configurable (catalog.excerpt_chars, replacing a hardcoded slice(0, 200) that no documentation mentioned); and returns advisory notes on write with the concrete numbers rather than accepting anything silently.

Also stated explicitly: do not split entries to meet the length. Per-entry overhead multiplies by count. Cut down into the deferred layer, not across into more records.

Search mode: the agent never fetched anything

A truncated catalog line was indistinguishable from a complete one, so an agent could not tell which entries it actually understood — and fetched neither. Cut lines now end in …[TRUNCATED] with a header counting them, and the absence of the marker is a guarantee the entry is complete as shown. The mode instructions were rewritten around that distinction, including an explicit do not fetch a complete line just to be thorough.

New surface

Command / tool Cost Purpose
axme-code kb-doctor [--fix], axme_kb_doctor free, instant Deterministic defect scan: slugs, leaked markup, budget overruns, duplicate titles. Exits 1 on findings — usable as a CI gate.
axme-code audit-kb --dry-run one LLM run Preview a compaction pass.
axme_archive_memory / axme_archive_decision free The missing half of the storage API.

On the archive tools: axme-code could create knowledge but not retire it, while the knowledge bases it builds carry a rule of their own — "write to axme-code storage via MCP tools only, never manually". An agent asked to clean up had no legal move, and the first real compaction had to bypass MCP with file operations. Archival is reversible (files move to .axme-code/archive/), marked (superseded/revoked written before the move), and never deletes. A superseded_by that does not resolve is refused rather than written as a dangling pointer.

audit-kb now also takes a backup before applying — .axme-code/ is gitignored by design (D-026), so a pass that rewrites every file had no safety net at all — and reindexes afterwards, since compaction rewrites the text the embeddings index was built from.

Verification

  • npm test — 657/657 pass, 44 new tests (test/kb-hygiene.test.ts plus rewrites in memory.test.ts / context.test.ts for deliberately changed behaviour).
  • npx tsc --noEmit clean, npm run build clean.
  • axme-code self-test — 7/7, including a new check that round-trips two Cyrillic titles and asserts both are readable back under distinct filenames.
  • Smoke-tested end to end on a synthetic base carrying every defect class: scan reports 4, --fix repairs 3 and leaves the judgment case, second --fix is a no-op, and the repaired record keeps every frontmatter field.

Known: test/audit-dedup.test.ts "ensureAxmeSessionForClaude - parallel processes (E2E)" fails intermittently. Verified failing on clean main via git stash — pre-existing, not from this branch.

Not done

Release tagging and publishing — per the standing prohibition, this PR stops at the version bump (0.6.4 in package.json, package-lock.json, .claude-plugin/plugin.json, templates/plugin-README.md) and CHANGELOG entry.

🤖 Generated with Claude Code

George-iam and others added 3 commits August 18, 2026 13:16
…format contract

Driven by a full manual compaction of a production knowledge base
(282 memories / 115 decisions, ~192k tokens at session start), which
surfaced one data-loss bug, one silent-failure bug, and a format contract
documented everywhere and enforced nowhere.

Fixed
- Non-Latin titles produced an empty slug, writing the memory as bare
  `.md` (a dotfile, invisible to shell globs) which the next such title
  overwrote. 4 affected files found on one project, 2 confirmed
  overwrites. Slugs now transliterate Cyrillic/Greek, strip Latin
  diacritics, hash-fallback when nothing survives, and prefix digit-only
  slugs. saveMemory refuses to overwrite a file whose stored title
  differs from the incoming one.
- audit-kb could exit 0 having written nothing: its counters were
  regex word-counts over the agent's prose. It now snapshots loaded-layer
  size before and after and reports the measured diff; a zero-change pass
  says so and exits 2.
- paginateSections never split an oversized section, so page 1 could
  render only a heading with all content on page 2.
- Leaked tool-call markup (`</description><parameter name=...>`) was
  persisted verbatim; storage now strips it and reports which fields.
- Frontmatter key replacement used `\s*`, which spans newlines and
  deleted the following field.

Added
- `axme-code kb-doctor [--fix]` + `axme_kb_doctor`: deterministic defect
  scan, no LLM. Exits 1 on findings.
- `axme_archive_memory` / `axme_archive_decision`: the missing half of
  the storage API. Reversible, marked, never deletes.
- `catalog.excerpt_chars` / `catalog.size_warn` config, replacing a
  hardcoded slice(0, 200) that no documentation mentioned.
- `audit-kb --dry-run`, automatic pre-apply backup, storage self-repair
  at session start, KB hygiene reporting in axme_context.

Changed
- CLAUDE.md template, Cursor rules, server instructions and tool
  descriptions carry a selection test with an explicit NEGATIVE list.
  "Save every successful approach" with no counterpart is what produced
  110 research diaries and 18 handoffs in one base.
- The two-level format (loaded description vs deferred body) is stated
  as a contract. `## Details` was non-empty in 11 of 126 memories: the
  mechanism was right and simply never explained.
- Save tools return advisory notes (overrun with concrete numbers, merge
  candidates, no-op title dedup) instead of accepting anything silently.
- Search-mode catalog marks cut entries `…[TRUNCATED]`; absence of the
  marker guarantees the entry is complete as shown.

Tests: 657/657 pass (+44 new). One pre-existing flaky E2E in
audit-dedup.test.ts fails intermittently on clean main too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tion

Validating the repair against the two affected records in axme-code's own
knowledge base showed the leak sits at the END of the description, with
`## Details` directly below it. `stripLeakedMarkup` cuts to the end of the
string — correct for a single field value, where everything past the frame
belongs to a different argument, but catastrophic applied to a whole
record: it would have destroyed the deferred layer this release exists to
protect.

Record-level repair now truncates the offending line at the tag, drops
only the frame-only lines trailing it, and keeps everything from the first
real line onward. Field-level stripping on the save path is unchanged.

Verified against a copy of the real damaged record: markup gone,
description intact, `## Details` preserved. Three new tests.

Tests: 660/660 (one pre-existing flaky E2E excluded).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The largest remaining gap. The guidance had been fixed in the CLAUDE.md
template and the MCP tool descriptions — i.e. the paths a human-directed
agent uses — while the three paths that write MOST entries were untouched:

- session-auditor.ts: the JSON schema did not include `body` at all, so
  the deferred layer was literally unreachable from the automatic path.
- memory-extractor.ts: told the model `body: Keep short or omit —
  description must carry all meaning`. That is the exact inversion of the
  contract, and a direct contributor to 91% of one base's memories sitting
  entirely in the layer paid for by every session.
- axme_begin_close checklist: no negative list, no format rule.

All three now state the selection test, the two-level format quoting the
project's real catalog.excerpt_chars, and the meta-decision prohibition.
Shared text lives in src/storage/kb-format.ts so five surfaces cannot
drift apart again.

Also closes three spec items that were still open:
- axme_merge_memories: agent composes the merged text, tool rewrites the
  survivor and archives the rest. Refuses if any source is missing.
- Meta-decisions refused at code level, not just discouraged. Matched on
  the title only, so a decision citing another in its body still saves.
- KB-audit counter surfaced in axme_context. It was written to the stderr
  of a detached background worker, which nobody reads.

Tests: 666/666 (+5). One pre-existing flaky E2E excluded.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant