From ebd64916f13cead4612b3b7a9b14afcf0c3bc4d5 Mon Sep 17 00:00:00 2001 From: Polichinl Date: Wed, 2 Sep 2026 22:15:02 +0200 Subject: [PATCH 1/2] docs: resolve the corpus's 3,300 cross-references, and stop the allowlist drifting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 0 of the governance consistency review. The documents cite each other about 3,300 times — 1,165 ADR references, 1,829 register concerns, 125 disagreements, 179 source paths. Every one is a claim that something exists. None was checked, and the only reason none had rotted is that nobody had renamed a file yet. `scripts/check_doc_refs.py` resolves each against its source of truth and runs in the `docs` CI job beside `check_arch_tree.py`. Offline by design: issue references need the network, and a doc check that goes red when GitHub does is a doc check people learn to ignore. `--check-issues` covers them manually. **The allowlist is the load-bearing part, and it is guarded in both directions.** Seventeen references are correct despite not resolving locally — a sibling repo's ADR, a concern in another register, an id this register skipped on purpose. A checker that flags those gets switched off within a week. So each is listed with its reason, AND the script fails if an id it allowlists is not also named in the register's Register Conventions. Neither half can quietly diverge from the other. Writing that guard immediately found that the register documented **8 of 17** non-local ids; nine were cited in the corpus and explained nowhere. That is the same drift the script exists to catch, in the convention that governs it. The register now carries the full list. **Two findings the first run produced, both about the checker rather than the docs.** `perspectives/` and `critiqus/` are written from OTHER repositories' points of view — a datafactory reviewer citing datafactory's ADRs, concerns and source tree. Resolving those against this register flagged 36 pieces of correct prose. References there are the author's namespace and are not resolved; paths under `src/views_frames*` still are, because those are unambiguously about us. `critiqus/` — 4 files, 1,194 lines — was missing from the corpus survey this review was scoped on. The real corpus is 100 markdown files, not 82. Mutation-tested six ways, all firing: a dangling ADR; a dangling concern; a dangling path under our own packages; removing a foreign id from the allowlist; undocumenting one in the register; and a missing register. The last of those found a defect in the script itself. A moved register produced an unhandled FileNotFoundError traceback rather than a named failure — the same self-disabling shape C-89 records, in a script written to prevent that class. It now says which file it could not find and exits 1. Two of the six mutations initially "passed" because I aimed them at a file that did not contain the reference. A no-op mutation proves nothing; they were redone against CLAUDE.md and docs/CICs/SpatioTemporalIndex.md, which do. What it does not catch, stated in the script: that a reference points at the RIGHT thing. ADR-018 resolving proves the file exists, not that it says what the citing sentence claims. That is a comparison problem, and comparison is what the read in Phase 1 is for. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 4 + reports/technical_risk_register.md | 12 ++ scripts/check_doc_refs.py | 333 +++++++++++++++++++++++++++++ 3 files changed, 349 insertions(+) create mode 100644 scripts/check_doc_refs.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2e97516..a51d3a3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -66,6 +66,10 @@ jobs: - uses: actions/checkout@v4 - run: bash docs/validate_docs.sh - run: python3 scripts/check_arch_tree.py + # Cross-reference resolution: ~3,300 ADR/concern/disagreement/path citations across the + # governance corpus, none of which was checked before 2026-09-02. Offline by design — + # issue references need the network and are left to a manual `--check-issues` sweep. + - run: python3 scripts/check_doc_refs.py # Formatting, kept out of the four-version matrix for the same reason as the # docs job: `ruff format` does not depend on the Python version, so running it diff --git a/reports/technical_risk_register.md b/reports/technical_risk_register.md index 72af88d..a08fda1 100644 --- a/reports/technical_risk_register.md +++ b/reports/technical_risk_register.md @@ -1943,6 +1943,18 @@ A spatial-forecasting showcase with no spatial display under-serves the audience - **Skipped ids:** **C-04** was merged into C-18 (the "SpatialLevel slippery slope"). **C-30** is intentionally skipped — it is *pipeline-core's* external id for the cross-repo contract-test gap (referenced in ADR-005 / ADR-016), not a views-frames concern. **C-48** is intentionally skipped — it is *views-reporting's* external id for the run-identity concern (referenced in D-02 / ADR-020), not a views-frames concern. - **Foreign ADR references:** an unqualified `ADR-xxx` always means *this* repository's ADR. A sibling repo's ADR must name the repo ("views-datafactory's ADR-044 **there**"). Three currently referenced numbers — **ADR-034** (pipeline-core), **ADR-044** (views-datafactory), **ADR-055** (paired with a `D-29` that does not exist here) — have no file in `docs/ADRs/`, which is correct, but only one of the three said so plainly. Same rule as the concern-id convention below. - **Foreign ids (collisions, not skips):** unlike the skipped ids above, **C-65** exists in *both* registers — pipeline-core's C-65 is the reversed entity-first tuple (cited in **C-18**), while *this* register's C-65 is the non-finite fail-loud blocked-path gap (resolved 2026-06-28). Any cross-register id must name its repo; an unqualified `C-xx` always means this register. +- **The full non-local id list (added 2026-09-02).** The two conventions above named the ids + that had come up in conversation; a cross-reference sweep found nine more already cited in + this corpus and documented nowhere, which is the same shape as the drift they exist to + prevent. `scripts/check_doc_refs.py` holds the machine-readable copy and **fails if an id it + allowlists is not also named here**, so the two halves cannot diverge again. Foreign + concerns: **C-30**, **C-48**, **C-108**, **C-135**, **C-164**, **C-165**, **C-167**, + **C-184**, **C-186** (views-reporting) and **C-198** (views-pipeline-core). Foreign + disagreements: **D-28**, **D-33** (views-pipeline-core) and **D-29** (paired with ADR-055). + Skipped local id: **C-04**. Note that `perspectives/` and `critiqus/` are written from other + repositories' points of view — ids and paths in those documents are the *author's* namespace + and are deliberately not resolved against this register. + - **Causal clusters** (assigned by `review-rr`, last reviewed **2026-08-17**). This list is the **single authority** on clustering — the Open-section preamble points here and must not restate it: - **doc↔code topology drift** = {**C-86**; resolved C-82, C-84; + resolved C-09 as the origin, C-39, C-23, C-70} — **REOPENED 2026-08-17, same day it was closed.** S1 #241 and S2 #242 closed C-82 and C-84, and the closure note said those two were "the whole of it". One story later, S3's sweep for documents restating what it was correcting found a **third** stale shape-claim — the `README.md` directory tree, missing 11 of 36 modules, plus a §9 pointer sending consumers to a `tests/conformance/` path that does not exist (C-86). The correction is recorded rather than quietly amended, because a cluster declaring itself closed while a member remains is the same failure the cluster is about. *The documents that describe the system's shape were never re-verified against it.* ADR-002 and `docs/standards/physical_architecture_standard.md` both describe an intended structure that the code moved past: `io/` on top importing the frames (the code is the inverse), a directory tree missing three shipped modules and containing two that never shipped, and one of three packages. The origin is datable — **C-09**, resolved 2026-06-21, moved `io/` onto a generic state-dict contract and inverted the dependency; neither topology document was amended, and `Persistable` (which puts `save`/`load` on the frame) makes the code's direction the only one available under the ADR-018 freeze. **The two entries were one editing session**, and were sequenced as one: the C-82 amendment touched `physical_architecture_standard.md:48-49,:67`, exactly where C-84's rewrite started, so S2 was blocked on S1 rather than run beside it. The standard now carries a `Last reviewed` date, a perishability note, and `scripts/check_arch_tree.py`, which makes the next drift detectable in one command. Distinguished from the cluster below by *what* is unverified: here it is a claim about structure, there it is a claim about coverage. - **unchecked completeness claims** = {C-85, C-77, C-80; + resolved C-64, C-74, C-75, C-81, C-51, C-67} — **an artifact asserts something about its own coverage or result, and nothing checks the assertion.** `docs/CICs/README.md` has claimed "fully contracted" wrongly three times (C-64 `Reconcile.md`, C-81 `Conformance.md`, now `FrameMetadata` in C-85); `GOVERNANCE.md` names three of six published conformance exports; ADR-018 inventories a frozen surface that omits `feature_names`; resolution fields described intent rather than result four times in one epic (C-77); the test suite's self-description does not match its contents (C-80). C-81's own resolution text is the tell: it was *"found only because this claim of completeness was audited against the code."* **The remedy is one mechanical change, not five edits:** `docs/validate_docs.sh` already runs in CI (C-74) and already checks placeholders, dangling references and the version banner — it checks no enumeration. Three assertions would have caught four of these findings automatically (every `__all__` name appears in its CIC; every public class has a CIC or a listed exemption; `GOVERNANCE.md`'s conformance names match `conformance.__all__`). Correcting the lists without the script edit schedules the fourth instance. **Tier within this cluster follows who reads the claim:** an external reader (a consumer running the floor, a future maintainer executing C-66's MAJOR instructions) → Tier 3; an internal auditor → Tier 4. That is why C-85 and C-77 are 3 while C-80 is 4, and the rule should be applied to any entry joining this cluster. diff --git a/scripts/check_doc_refs.py b/scripts/check_doc_refs.py new file mode 100644 index 0000000..dc0df01 --- /dev/null +++ b/scripts/check_doc_refs.py @@ -0,0 +1,333 @@ +"""Cross-reference resolution for the governance corpus. + +The documents in this repository cite each other about 3,600 times — roughly 1,160 `ADR-NNN` +references, 1,770 `C-NN` register concerns, 120 `D-NN` disagreements and 140 source paths in +prose. Every one is a claim that something exists. Until this script, none of them was checked, +and the only reason none had rotted is that nobody had renamed a file yet. + +This resolves each reference against its source of truth: + + ADR-NNN -> docs/ADRs/NNN_*.md exists + C-NN -> a `### C-NN` header in the register + D-NN -> a `### D-NN` header in the register + src/... .py -> the file exists + tests/....py -> the file exists + +**The allowlist is the load-bearing part.** Some references are deliberately non-local: a +sibling repository's ADR, a concern in another repo's register, an id this register skipped on +purpose. A checker that flags those is a checker someone switches off within a week, so each +one is listed below with its reason. + +**The allowlist is not free-floating.** `reports/technical_risk_register.md` already documents +every non-local id in its *Register Conventions* section — that prose is the authority, and +this script is the machine-readable half. To stop the two drifting apart (which is the exact +defect class this script exists to catch), every allowlisted id must also appear in that +section, and the script fails if one does not. Adding a foreign id here without documenting it +there is therefore an error, in both directions. + +**What this does not catch.** That a reference points at the *right* thing. `ADR-018` resolving +to a file proves the file exists, not that ADR-018 says what the citing sentence claims. That +is a comparison problem, and comparison is what a human read is for. + +GitHub issue references (`#NNN`) are **not** resolved by default. CI has no business calling +the GitHub API, and a doc check that needs the network is a doc check that goes red when +GitHub does. Use ``--check-issues`` for a manual sweep. + +Usage:: + + python3 scripts/check_doc_refs.py # CI: offline, exits 1 on a dangling ref + python3 scripts/check_doc_refs.py --check-issues # manual: also resolves #NNN via `gh` + +Exits 0 when every reference resolves or is allowlisted, 1 with a report when it does not. +""" + +from __future__ import annotations + +import re +import subprocess +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +REGISTER = REPO_ROOT / "reports" / "technical_risk_register.md" + +# Directories whose contents are not governance prose. +SKIP_DIRS = { + ".git", + "graphify-out", + "node_modules", + ".venv", + "__pycache__", + ".mypy_cache", +} + +# --- The allowlist ------------------------------------------------------------------------- +# +# Every entry is a reference that is CORRECT despite not resolving locally. The reason is not +# decoration: it is what lets the next reader tell "foreign" from "broken" without going and +# finding out. Each id must also be documented in the register's Register Conventions section +# — `_check_allowlist_is_documented` enforces that, so this list cannot quietly diverge. + +FOREIGN_ADRS = { + "034": "views-pipeline-core's ADR", + "044": "views-datafactory's ADR (area-majority GAUL work)", + "055": "sibling repo's ADR, paired with the foreign D-29", +} + +FOREIGN_CONCERNS = { + "30": "views-pipeline-core's id for the cross-repo contract-test gap — skipped here on purpose", + "48": "views-reporting's concern (run-identity ambiguity)", + "108": "views-reporting's concern", + "135": "views-reporting's concern", + "164": "views-reporting's concern", + "165": "views-reporting's concern", + "167": "views-reporting's concern", + "184": "views-reporting's concern", + "186": "views-reporting's concern (the #181 report-stage OOM)", + "198": "views-pipeline-core's concern", +} + +# Ids this register deliberately never assigned. Distinct from foreign ids: these are *local* +# numbers that were skipped, and the register says why. +SKIPPED_CONCERNS = { + "04": "merged into C-18 (the SpatialLevel slippery slope)", +} + +FOREIGN_DISAGREEMENTS = { + "28": "views-pipeline-core's disagreement (relocate reconciliation)", + "29": "sibling repo's disagreement, paired with the foreign ADR-055", + "33": "views-pipeline-core's disagreement", +} + +# Paths that appear in prose as illustrations of something that does NOT exist — a file a +# future violation would create, or an elision. Flagging these would be flagging the prose for +# being explanatory. +HYPOTHETICAL_PATHS = { + "src/views_frames/metric_frame.py": "a hypothetical future file, cited to describe what a violation would look like", +} + +# Template files legitimately contain placeholder references. +TEMPLATE_FILES = {"docs/CICs/cic_template.md", "docs/ADRs/adr_template.md"} + +# Documents written from ANOTHER repository's point of view. `perspectives/` holds cross-repo +# design reviews ("from views-datafactory's perspective") and `critiqus/` holds the critique +# rounds that fed them. In these, an unqualified `ADR-045` or `C-182` is the *author's* repo's +# numbering, not ours, and `src/datafactory_adapters/...` is their tree. Resolving those against +# this repository would flag correct prose, which is how a check gets switched off. +# +# They are not exempt, only differently scoped: a path under THIS repo's own packages is still +# checked in them, because that reference is unambiguously about us. +CROSS_REPO_DIRS = ("perspectives/", "critiqus/") + +# Paths that are unambiguously this repository's, wherever they appear. +OWN_SOURCE_PREFIXES = ("src/views_frames",) + +ADR_RE = re.compile(r"\bADR-(\d{3})\b") +CONCERN_RE = re.compile(r"\bC-(\d{2,3})\b") +DISAGREEMENT_RE = re.compile(r"\bD-(\d{2})\b") +PATH_RE = re.compile(r"\b((?:src|tests|scripts)/[A-Za-z0-9_./-]+\.py)\b") +ISSUE_RE = re.compile(r"(? list[Path]: + """Every governance document in the repository, in a stable order.""" + out = [] + for p in sorted(REPO_ROOT.rglob("*.md")): + if any(part in SKIP_DIRS for part in p.parts): + continue + out.append(p) + return out + + +def local_adrs() -> set[str]: + return { + p.name.split("_")[0] for p in (REPO_ROOT / "docs" / "ADRs").glob("[0-9]*.md") + } + + +def register_ids(prefix: str) -> set[str]: + """Every `### C-NN` / `### D-NN` header in the register, normalised without leading zeros. + + Raises FileNotFoundError if the register has moved. `main` turns that into a loud, named + failure rather than a traceback: a check that dies obscurely when its input moves is the + same defect as one that silently passes (register C-89). + """ + text = REGISTER.read_text() + return { + m.group(1).lstrip("0") or "0" + for m in re.finditer(rf"^### {prefix}-(\d+):", text, re.M) + } + + +def _norm(n: str) -> str: + return n.lstrip("0") or "0" + + +def _check_allowlist_is_documented() -> list[str]: + """Every allowlisted id must be named in the register's conventions prose. + + This is the half that stops the allowlist becoming a second, silently diverging source of + truth — the failure mode the register calls "the instance fix instead of the class fix". + """ + errors = [] + if not REGISTER.exists(): + return [ + f"ERROR: {REGISTER} is missing; cannot verify the allowlist is documented" + ] + conventions = REGISTER.read_text() + marker = "Register Conventions" + if marker in conventions: + conventions = conventions[conventions.index(marker) :] + for num in FOREIGN_ADRS: + if f"ADR-{num}" not in conventions: + errors.append( + f"ERROR: allowlisted ADR-{num} is not documented in the register's {marker}" + ) + for num in {**FOREIGN_CONCERNS, **SKIPPED_CONCERNS}: + if f"C-{num}" not in conventions: + errors.append( + f"ERROR: allowlisted C-{num} is not documented in the register's {marker}" + ) + for num in FOREIGN_DISAGREEMENTS: + if f"D-{num}" not in conventions: + errors.append( + f"WARN: allowlisted D-{num} is not named in the register's {marker} — " + "document it there so a reader can tell foreign from broken" + ) + return errors + + +def _resolve_issues(numbers: set[str]) -> list[str]: + """Manual-only: ask `gh` whether each issue number exists. Never called from CI.""" + errors = [] + for num in sorted(numbers, key=int): + r = subprocess.run( + ["gh", "issue", "view", num, "--json", "number"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + if r.returncode != 0: + r = subprocess.run( + ["gh", "pr", "view", num, "--json", "number"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + if r.returncode != 0: + errors.append( + f"ERROR: #{num} resolves to neither an issue nor a pull request" + ) + return errors + + +def main() -> int: + check_issues = "--check-issues" in sys.argv + errors: list[str] = [] + errors.extend(_check_allowlist_is_documented()) + + adrs = local_adrs() + try: + concerns = register_ids("C") + disagreements = register_ids("D") + except FileNotFoundError: + print( + f" ERROR: {REGISTER.relative_to(REPO_ROOT)} not found; cannot resolve any C-/D- reference" + ) + return 1 + if not adrs or not concerns: + print( + " ERROR: found no local ADRs or no register concerns; the check cannot be vacuously true" + ) + return 1 + + counts = {"ADR": 0, "C": 0, "D": 0, "path": 0, "issue": 0} + issue_numbers: set[str] = set() + + for doc in markdown_files(): + rel = doc.relative_to(REPO_ROOT).as_posix() + is_template = rel in TEMPLATE_FILES + cross_repo = rel.startswith(CROSS_REPO_DIRS) + text = doc.read_text(errors="replace") + + for m in ADR_RE.finditer(text): + num = m.group(1) + counts["ADR"] += 1 + if num in adrs or num in FOREIGN_ADRS or is_template or cross_repo: + continue + errors.append( + f"ERROR: {rel} cites ADR-{num}, which has no file in docs/ADRs/" + ) + + for m in CONCERN_RE.finditer(text): + num = _norm(m.group(1)) + counts["C"] += 1 + if cross_repo or num in concerns: + continue + if num in {_norm(k) for k in {**FOREIGN_CONCERNS, **SKIPPED_CONCERNS}}: + continue + errors.append( + f"ERROR: {rel} cites C-{m.group(1)}, which has no entry in the register" + ) + + for m in DISAGREEMENT_RE.finditer(text): + num = _norm(m.group(1)) + counts["D"] += 1 + if cross_repo or num in disagreements: + continue + if num in {_norm(k) for k in FOREIGN_DISAGREEMENTS}: + continue + errors.append( + f"ERROR: {rel} cites D-{m.group(1)}, which has no entry in the register" + ) + + for m in PATH_RE.finditer(text): + path = m.group(1) + # Only OUR paths are resolvable. `src/datafactory_adapters/...` in a cross-repo + # review is that repo's tree; `tests/...` there is ambiguous, so it is checked only + # in local documents where `tests/` can only mean ours. + ours = path.startswith(OWN_SOURCE_PREFIXES) or ( + not cross_repo and path.startswith(("tests/", "scripts/")) + ) + if not ours: + continue + counts["path"] += 1 + if path in HYPOTHETICAL_PATHS or (REPO_ROOT / path).exists(): + continue + errors.append(f"ERROR: {rel} cites `{path}`, which does not exist") + + if check_issues: + for m in ISSUE_RE.finditer(text): + counts["issue"] += 1 + issue_numbers.add(m.group(1)) + + if check_issues: + errors.extend(_resolve_issues(issue_numbers)) + + total = sum(counts.values()) + print(f"references checked: {total}") + for kind, n in counts.items(): + if n: + print(f" {kind:6s} {n}") + allow = ( + len(FOREIGN_ADRS) + + len(FOREIGN_CONCERNS) + + len(SKIPPED_CONCERNS) + + len(FOREIGN_DISAGREEMENTS) + ) + print(f" allowlisted non-local ids: {allow} (each documented in the register)") + if not check_issues: + print(" (#NNN issue references not resolved — run with --check-issues)") + + if errors: + print() + for e in errors: + print(f" {e}") + print(f"\nFAILED: {len(errors)} unresolved reference(s)") + return 1 + print("\nPASSED: every reference resolves or is a documented non-local id") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From efb250166cace14b52cefa72309006071c9c153b Mon Sep 17 00:00:00 2001 From: Polichinl Date: Wed, 2 Sep 2026 22:39:07 +0200 Subject: [PATCH 2/2] docs: hold prose counts, CIC test references and amendments against the filesystem (closes C-98) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 0 of the governance consistency review, second half. Three checks added to `docs/validate_docs.sh`, and the four stale counts they immediately caught. **Check 11 — counts stated in prose.** Three documents stated numbers derived from the document set, and they were wrong DIFFERENTLY from each other: CLAUDE.md "CICs ... (7 active)" 9 exist CLAUDE.md "project ADRs 011-026" 011-029 docs/ADRs/README.md "Project-Specific ADRs (011-027)" 011-029 (x2) Two documents counting the same range, disagreeing with each other AND with the filesystem, is worse than either being wrong alone — a reader who checks one against the other finds a contradiction with no way to tell which is right. A third defect surfaced while fixing them. CLAUDE.md said "Constitutional ADRs 000-010", folding ADR-010 into the constitutional range. The ADR index has three sections, not two: constitutional 000-009, a Governance ADRs section holding only ADR-010, then project 011-029. The index's structure is the authority. Check 11 pins the CIC count, every project-ADR range against the highest ADR on disk, and the register's header against its own body. It validated the C-98 entry this commit adds, seconds after being written. **Check 12 — CIC Test Alignment references.** Each CIC's section 10 cites the tests holding its guarantees; that citation is the only link between a contract and its evidence, and nothing stopped a rename stranding one. All 19 references resolve, and the check verifies both the file and, where named, the function inside it. **Check 13 — amendments are two-sided.** When ADR-A carries "Amendment (date, ADR-B)", ADR-B must mention ADR-A. A one-sided amendment is how a reader arrives at the amended document and never learns it was amended. The ADR-018 / ADR-025 / ADR-028 triangle was correct by hand; it is now checked. Mutation-tested, and the harness matters here: **two of my mutations initially "passed" because I aimed them at files that did not contain the string** — the same no-op mistake made earlier in this review. `test_frame_parity.py` lives in TargetFrame.md, not PredictionFrame.md. The rerun asserts the file actually changed before reading any result, and only then are the results meaningful: a drifted CIC count, a new ADR with ranges unupdated, a register header that stops matching its body, a renamed test file, and a vanished test function all error. What check 11 does not catch, stated in its comment: a count stated in a phrasing the pattern does not match. It targets three known claim shapes. A general numeric auditor would flag every number in the corpus and be switched off within a week. Register: 95 entries, 10 open, 85 resolved. Still 0 actionable. Co-Authored-By: Claude Opus 5 (1M context) --- CLAUDE.md | 5 +- docs/ADRs/README.md | 4 +- docs/validate_docs.sh | 125 +++++++++++++++++++++++++++++ reports/technical_risk_register.md | 46 ++++++++++- 4 files changed, 174 insertions(+), 6 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 94a1eab..2ace737 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -127,8 +127,9 @@ CI additionally gates 100% line+branch coverage ## Governance -Constitutional ADRs 000–010, project ADRs 011–026, CICs for every non-trivial -surface (7 active incl. the package-level `Summarize.md` and `Reconcile.md`), +Constitutional ADRs 000–009, the governance ADR 010, project ADRs 011–029, CICs for +every non-trivial surface (9 active incl. the package-level `Summarize.md` and +`Reconcile.md`), contributor protocols, and standards live in `docs/`. The technical risk register (`reports/technical_risk_register.md`) is the curated concern/decision log. Run `bash docs/validate_docs.sh` to check documentation consistency. Build *against* diff --git a/docs/ADRs/README.md b/docs/ADRs/README.md index e6207ed..3fd62b3 100644 --- a/docs/ADRs/README.md +++ b/docs/ADRs/README.md @@ -11,7 +11,7 @@ ADRs are divided into: 1. **Constitutional ADRs (000–009)** — foundational architectural rules. 2. **Governance ADRs (010)** — the technical risk register. -3. **Project-Specific ADRs (011–027)** — the ratified contract, estimator, sibling-package, +3. **Project-Specific ADRs (011–029)** — the ratified contract, estimator, sibling-package, freeze/immutability, and surface-scope decisions. --- @@ -51,7 +51,7 @@ Together, these define the invariant layer of the system. --- -## Project-Specific ADRs (011–027) — the ratified project decisions +## Project-Specific ADRs (011–029) — the ratified project decisions **011–016** ratify the six founding contract decisions from the design bible (README §13a, all Accepted 2026-06-21); **017–027** are the post-v1 decisions — sibling packages, the API diff --git a/docs/validate_docs.sh b/docs/validate_docs.sh index 5d659d7..9626e40 100755 --- a/docs/validate_docs.sh +++ b/docs/validate_docs.sh @@ -341,6 +341,131 @@ fi [ "$errors" -eq "$before" ] && echo " OK (wheel ships $pkg_count packages; checked $(printf '%s' "$WHEEL_DOCS" | wc -w) wheel-describing documents)" +# 11. Counts stated in prose match what is on disk. +# +# Three documents state numbers derived from the doc set itself — how many active CICs +# exist, which range the project ADRs span, how the register's totals add up. Every one is +# a hand-maintained copy of something countable, and on 2026-09-02 two of them were wrong +# AND wrong differently from each other: CLAUDE.md said "7 active" CICs against 9 on disk +# and "project ADRs 011-026", while the ADR index said "011-027", both against 029. Two +# copies of one number drifting independently is the signature of no shared source. +echo "--- Checking counts stated in prose against the filesystem ---" +before=$errors +cic_actual=$(ls CICs/*.md 2>/dev/null | grep -vE 'README|template' | wc -l | tr -d ' ') +adr_max=$(ls ADRs/[0-9]*.md 2>/dev/null | sed 's|.*/||;s/_.*//' | sort -n | tail -1) +if [ -z "$cic_actual" ] || [ "$cic_actual" -eq 0 ] || [ -z "$adr_max" ]; then + echo " ERROR: could not count CICs or ADRs on disk; the check cannot be vacuously true" + errors=$((errors + 1)) +else + claimed_cic=$(grep -oE '\(([0-9]+) active' ../CLAUDE.md 2>/dev/null | grep -oE '[0-9]+' | head -1) + if [ -n "$claimed_cic" ] && [ "$claimed_cic" != "$cic_actual" ]; then + echo " ERROR: CLAUDE.md claims $claimed_cic active CICs; $cic_actual exist" + errors=$((errors + 1)) + fi + # Every "project ADRs NNN-MMM" style range must end at the highest ADR on disk. + while IFS=: read -r file line; do + [ -z "$file" ] && continue + end=$(echo "$line" | grep -oE '0[0-9]{2}[^0-9]{1,3}0[0-9]{2}' | tail -1 | grep -oE '0[0-9]{2}$') + [ -z "$end" ] && continue + if [ "$end" != "$adr_max" ]; then + echo " ERROR: $file states a project-ADR range ending $end; the highest ADR is $adr_max" + errors=$((errors + 1)) + fi + done </dev/null | sed 's/^\([^:]*\):[0-9]*:/\1:/') +EOF + # The register's header must agree with its own body. + reg=../reports/technical_risk_register.md + if [ ! -f "$reg" ]; then + echo " ERROR: no $reg; cannot check its header arithmetic" + errors=$((errors + 1)) + else + hdr_total=$(grep -m1 '^| Total Concerns' "$reg" | grep -oE '[0-9]+') + hdr_open=$(grep -m1 '^| Open Concerns' "$reg" | grep -oE '[0-9]+') + hdr_res=$(grep -m1 '^| Resolved Concerns' "$reg" | grep -oE '[0-9]+') + body_total=$(grep -cE '^### C-[0-9]+' "$reg") + body_open=$(grep -E '^### C-[0-9]+' "$reg" | grep -vc RESOLVED) + if [ "$hdr_total" != "$body_total" ]; then + echo " ERROR: register header says $hdr_total concerns; $body_total entries exist" + errors=$((errors + 1)) + fi + if [ "$hdr_open" != "$body_open" ]; then + echo " ERROR: register header says $hdr_open open; $body_open entries are not RESOLVED" + errors=$((errors + 1)) + fi + if [ "$((hdr_open + hdr_res))" != "$hdr_total" ]; then + echo " ERROR: register header does not add up: $hdr_open open + $hdr_res resolved != $hdr_total" + errors=$((errors + 1)) + fi + fi +fi +[ "$errors" -eq "$before" ] && echo " OK ($cic_actual active CICs, ADRs to $adr_max, register header reconciles)" + +# 12. Every test file a CIC names in its Test Alignment section still exists. +# +# Each CIC's Section 10 cites the tests that hold its guarantees. Those citations are the +# only link between a contract and its evidence, and nothing stopped a rename stranding +# one — a CIC would keep claiming a test that no longer exists, which is worse than +# claiming none. +echo "--- Checking test files named in CIC Test Alignment sections ---" +before=$errors +ref_count=0 +while read -r ref; do + [ -z "$ref" ] && continue + ref_count=$((ref_count + 1)) + file="../${ref%%::*}" + if [ ! -f "$file" ]; then + echo " ERROR: a CIC names $ref, but that test file does not exist" + errors=$((errors + 1)) + continue + fi + case "$ref" in + *::*) + fn="${ref##*::}" + grep -qE "^def ${fn}\(" "$file" || { + echo " ERROR: a CIC names $ref, but $file defines no $fn" + errors=$((errors + 1)) + } ;; + esac +done </dev/null | sort -u) +EOF +if [ "$ref_count" -eq 0 ]; then + echo " ERROR: found no test references in any CIC; the check cannot be vacuously true" + errors=$((errors + 1)) +fi +[ "$errors" -eq "$before" ] && echo " OK (checked $ref_count test references named in CICs)" + +# 13. An amendment is declared on both sides. +# +# When ADR-A carries "Amendment (date, ADR-B)", ADR-B must mention ADR-A. A one-sided +# amendment is how a reader arrives at the amended document and never learns it was +# amended. The ADR-018 / ADR-025 / ADR-028 triangle was correct by hand; this keeps it so. +echo "--- Checking amendment declarations are two-sided ---" +before=$errors +amend_count=0 +for adr in ADRs/[0-9]*.md; do + self=$(basename "$adr" | sed 's/_.*//') + while read -r other; do + [ -z "$other" ] && continue + amend_count=$((amend_count + 1)) + target=$(ls ADRs/${other}_*.md 2>/dev/null | head -1) + if [ -z "$target" ]; then + echo " ERROR: $adr names an amendment by ADR-$other, which has no file" + errors=$((errors + 1)) + continue + fi + grep -qE "ADR-0*${self#0}|ADR-${self}" "$target" || { + echo " ERROR: $adr says it was amended by ADR-$other, but ADR-$other never mentions ADR-$self" + errors=$((errors + 1)) + } + done </dev/null | grep -oE '[0-9]{3}$' | sort -u) +EOF +done +[ "$errors" -eq "$before" ] && echo " OK (checked $amend_count amendment declaration(s))" + + echo "" if [ "$errors" -gt 0 ]; then echo "=== FAILED: $errors issue(s) found ===" diff --git a/reports/technical_risk_register.md b/reports/technical_risk_register.md index a08fda1..a25b243 100644 --- a/reports/technical_risk_register.md +++ b/reports/technical_risk_register.md @@ -5,9 +5,9 @@ | Project | views-frames | | Owner | VIEWS platform maintainers | | Last Updated | 2026-08-18 | -| Total Concerns | 94 | +| Total Concerns | 95 | | Open Concerns | 10 | -| Resolved Concerns | 84 | +| Resolved Concerns | 85 | | Disagreements | 12 | --- @@ -441,6 +441,48 @@ Cross-refs: C-47 (eval provenance kept out of the generic header — the precede > Resolved 2026-07-31 by **ADR-027** (Epic #208 / S1 #209) — the #113 decision. +### C-98: counts stated in prose drifted from what is on disk — RESOLVED + +| Field | Value | +|-------|-------| +| ID | C-98 | +| Tier | 4 | +| Resolved | 2026-09-02 | +| Resolution | Four stale counts corrected, and `docs/validate_docs.sh` check 11 added so prose counts are held against the filesystem. | +| Source | governance consistency review, Phase 0 (2026-09-02) | +| Cross-refs | C-96 (the wheel package count — the same class, one document over), C-70 (the README banner epoch-lag that produced check 6), C-84/C-86 (the stale trees), the *unchecked completeness claims* cluster. | + +Three documents stated numbers derived from the document set itself, and they were wrong +**differently from each other** — which is the signature of hand-maintained copies with no +shared source: + +| Site | Claimed | Actual | +|---|---|---| +| `CLAUDE.md` | "CICs … (**7 active**)" | **9** | +| `CLAUDE.md` | "project ADRs **011–026**" | **011–029** | +| `docs/ADRs/README.md` (×2) | "Project-Specific ADRs (**011–027**)" | **011–029** | + +Two documents counting the same range, disagreeing with each other *and* with the filesystem, +is worse than either being wrong alone: a reader who checks one against the other finds a +contradiction with no way to tell which is right. + +**A third defect surfaced while fixing them.** `CLAUDE.md` described the ADR set as +"Constitutional ADRs 000–010, project ADRs 011–026", folding **ADR-010** into the +constitutional range. The ADR index has three sections, not two — constitutional 000–009, a +**Governance ADRs** section containing only ADR-010, then project 011–029. The index's +structure is the authority; `CLAUDE.md` now matches it. + +Check 11 pins the active-CIC count, every "project ADRs NNN–MMM" range against the highest ADR +on disk, and the register's own header against its body (total, open, and that open + resolved +sums to total). Mutation-tested: a drifted CIC count, a new ADR landing with ranges unupdated, +and a header that stops matching its body all error. + +**What it does not catch:** a count stated in a phrasing the pattern does not match. This is a +targeted check on three known claim shapes, not a general numeric auditor — a general one would +flag every number in the corpus and be switched off within a week. + +--- + ### C-97: the status banners asserted a release state the repo cannot verify — RESOLVED | Field | Value |