You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Summary — ratchet bolt (b), slice 2 of 2: the row that makes the baseline unforgeable
Stacked on #766 (base branch wt/ratchet-b1-attest-store). Slice 1 built the store; this is the consumer. Merge #766 first, then retarget this PR to main before deleting that branch.
ratchet.yml gains actions: read and a step, Fetch the baseline attestation from main, that lists ratchet-attest.yml's runs on main through the Actions API, picks the newest successful push/workflow_dispatch run, downloads its ratchet-attestation artifact, and binds it to the run the API named — the artifact's run_id and main_sha must both match, or the hand-off is unresolved. Three retries on each API read, like the live-head read. Four outcomes, none silent: attested, bootstrap, unresolved; and an empty hand-off makes the collect step refuse to render.
The collector's new second row, baseline attestation, right after commit provenance ("which commit, then what it is measured against, then the numbers"). It diffs the PR's baseline fields (queries, latency_baseline_ms, thresholds) against the attestation, field by field:
unchanged → 🟢 naming the digest, run id, main sha and timestamp;
changed → 🔴 "baseline changed by hand; no CI attestation for these values", listing each field old → new;
changed to exactly what that main run measured (measured[path]) → 🟢 "moved to values measured by main". That is the legitimate path, and the only one.
If main has moved past the attestation's sha, the value says so (an attest run may still be in flight) — a note, not a verdict.
Fail closed. Missing, unparseable, wrong-schema, self-inconsistent, expired, or mis-bound artifact → 🔴. Two hand-off flags at once → 🔴. Asked to fetch and could not → 🔴 with the reason.
Bootstrap, narrowly. The API showing no successful attest run on main at all (this PR's own state: HTTP 404 — the writer is not on main yet) is the one case that is neither a measurement nor a finding. The row then compares against the base tip's committed fixture via git show, merge-ref only — a direct checkout's first parent is the PR's own branch and is refused — and says it is bootstrapping. Once one success exists on main this branch is never taken again: an expired artifact is 🔴 unresolved, refreshed by one workflow_dispatch.
The boundary, stated exactly (per #761's reviewer)
No INPUT to this row comes from the PR tree: the run list, the artifact and its binding are GitHub's. The comparator is this PR's checkout of ci_ratchet_table.py and could be rewritten to answer GREEN. Diff-reviewable, not tamper-proof. The docstring, the Notes column and the workflow comment say this and nothing wider.
Premise check — two corrections to the brief
Today no CI row compares a measured value to the baseline at all.mapped bytes, search p50/p95, idle CPU are fixture-bound n/a on every hosted runner; signature_valid compares counts to zero. The baseline is read by scripts/sprint_gate.py on the installed Mac and quoted in the table's Notes. So the harm was real (the installed gate and the reader-facing claim) but it was never a CI RED/GREEN flip.
measured is {} on every attestation today — no runner-side collector measures a baseline field. Consequence, stated in the row's Notes: today the baseline cannot move by PR at all. The key to this lock is a collector (a self-hosted installed-Mac runner, or a CI-measured count promoted into the baseline), not a hand — which is the ratchet's own rule. The legitimate path exists in code and is tested end to end through the CLI; it has nothing to carry until something is measured. Etan's call whether that posture is intended or a human-ratified override should exist.
Definition of done — proven against the old sources
Swapped 3ee7c279's scripts/ci_ratchet_table.py in and ran the four bolt tests:
Same four on this branch: 4 passed. Full file: 205 passed (plus slice 1's reader tests, kept).
The fetch step's bash was run locally against the live API (REPO=EtanHey/brainlayer) and produced bootstrap — ratchet-attest.yml is not a registered workflow on main yet (HTTP 404), which is what this PR's own table should render. The live row on this PR is the acceptance test; the report's DONE marker waits for it.
ruff check and ruff format --check on scripts/ and tests/
tests/test_ci_ratchet_table.py: 205 passed (row-level, real-git bootstrap on a merge ref, malformed artifacts, end-to-end CLI, pyyaml assertions on the fetch step — !cancelled() never always(), exactly one hand-off flag, retries, no reader of the checkout, actions: read and never write)
tests/test_kg_judge.py git-shelling registration: passes (_clean_git_env() on the new merge-ref repo helper)
Live: this PR's table renders baseline attestation 🟢 bootstrap and commit provenance 🟢 on the final head
Live, after both merges: the next PR's row reads matches the main attestation (run N · main <sha>)
Medium Risk
Changes CI gate semantics for baseline fixture edits and adds Actions API dependency on PR runs; mis-fetch or comparator bugs could block merges or miss hand edits, but scope is isolated to ratchet workflow/collector with extensive fail-closed tests.
Overview
Adds baseline attestation as the second ratchet row so sprint-gate baseline fields (queries, latency_baseline_ms, thresholds) are checked against a reference PRs cannot forge.
Workflow (ratchet.yml): Grants read-only actions permission and a new step that paginates successful ratchet-attest.yml runs on main, downloads ratchet-attestation, and verifies run_id/main_sha match the artifact. Outcomes hand off exactly one of --attestation, --attestation-bootstrap, or --attestation-unresolved (with retries and guarded jq so corrupt API bodies fail closed). Collect refuses to run without a non-empty attestation hand-off.
Collector (ci_ratchet_table.py): Compares the checked-out corpus baseline to the attested baseline; unlicensed edits are RED with an explicit hand-edit message, changes matching main’s measured map (type-sensitive) are GREEN, and pre-first-success on main uses merge-base git show bootstrap only. Stale attestations (artifact sha ≠ base tip) also diff against the base commit so reverting to an old attested value cannot pass silently.
Tests: Broad coverage for row logic, CLI exit codes, workflow YAML contracts, and executed bash for the fetch/collect steps.
Reviewed by Cursor Bugbot for commit 63e388c. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add baseline-attestation row to ratchet table so gate compares against main-measured values
The ratchet gate workflow fetches the latest successful ratchet-attest.yml artifact from main, verifies its run ID and SHA, and hands the artifact (or bootstrap/unresolved state) to the Python collector instead of letting the PR supply its own baseline.
scripts/ci_ratchet_table.py adds a baseline attestation row that reports GREEN when the PR baseline is unchanged or matches a value main measured, and RED for hand-edited fields, type-coerced values, or stale/reverted attestations. Bootstrap mode compares against the merge-ref base commit's baseline before any main attestation exists.
The licensed helper now requires exact value and type match for a baseline change to count as main-measured, preventing boolean/count coercion and dotted-key aliasing.
Risk: select_attestation fails closed for conflicting --attestation modes, malformed artifacts, and unresolved fetches; a missing or empty attestation hand-off now fails collection before rendering a table instead of rendering n/a.
…write (b, slice 1 of 2)
A new ratchet-attest.yml runs on every push to main (and a no-input
workflow_dispatch), runs the same collector with --attest-out, and publishes
what that main commit's baseline IS (the queries / latency_baseline_ms /
thresholds fields of corpus.json, with a canonical sha256) plus every value the
run measured, as the `ratchet-attestation` artifact. A pull_request run holds
contents: read and cannot upload into another run's artifacts, which is what
makes the artifact a reference a PR cannot edit. The writer refuses to attest a
commit it does not have checked out and refuses when any row is RED.
This slice is the writer, the schema and its reader (`read_attestation`, which
fails closed on every malformed shape). The row that compares a PR's baseline
against the artifact, and the ratchet.yml step that fetches it, are slice 2.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
… against a number the PR wrote (b, slice 2 of 2)
The collector's new second row, `baseline attestation`, diffs the PR's baseline
fields against the `ratchet-attestation` artifact of the latest successful
push/dispatch run of ratchet-attest.yml on main, which ratchet.yml now lists
through the Actions API (`actions: read`), downloads, and binds to the run the
API named (run id AND main sha must match). Unchanged is GREEN; a changed
field is RED "baseline changed by hand; no CI attestation for these values"
unless that main run measured the new value. Bootstrap -- no successful attest
run on main at all -- compares against the base tip's committed fixture via
`git show`, merge-ref only, and says so; once one success exists an expired
artifact is RED, never a fall-back. Boundary stated, not overclaimed: the
inputs are outside the PR tree, the comparator is not.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
We reviewed changes in 47133a9...63e388c on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.
Some issues found as part of this review are outside of the diff in this pull request and aren't shown in the inline review comments due to GitHub's API limitations. You can see those issues on the DeepSource dashboard.
AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.
Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.
…(Macroscope, #766)
A removed null-valued leaf is a change (path membership, not `.get()`); the
writer refuses run numbers the reader would reject and a checkout whose
baseline lacks a field; the reader requires exactly the baseline fields, so an
incomplete reference cannot pass as one.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
…ource, #766)
Flags first, checkout second, the same shape select_signature already has, so
adding a rule does not keep pushing one function's branch count up.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
…e a segment (Macroscope High, #767)
Slice 1 made baseline paths tuples; this is the row side: changes print dotted,
but the measured lookup refuses any path whose segment contains a dot, so the
(c)-facing dotted-string map cannot be aliased from the PR tree either.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
…write (b, slice 1 of 2) (#766)
* feat(ratchet): attest the baseline from main — the store a PR cannot write (b, slice 1 of 2)
A new ratchet-attest.yml runs on every push to main (and a no-input
workflow_dispatch), runs the same collector with --attest-out, and publishes
what that main commit's baseline IS (the queries / latency_baseline_ms /
thresholds fields of corpus.json, with a canonical sha256) plus every value the
run measured, as the `ratchet-attestation` artifact. A pull_request run holds
contents: read and cannot upload into another run's artifacts, which is what
makes the artifact a reference a PR cannot edit. The writer refuses to attest a
commit it does not have checked out and refuses when any row is RED.
This slice is the writer, the schema and its reader (`read_attestation`, which
fails closed on every malformed shape). The row that compares a PR's baseline
against the artifact, and the ratchet.yml step that fetches it, are slice 2.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
* fix(ratchet): writer and reader agree on what an attestation must be (Macroscope, #766)
A removed null-valued leaf is a change (path membership, not `.get()`); the
writer refuses run numbers the reader would reject and a checkout whose
baseline lacks a field; the reader requires exactly the baseline fields, so an
incomplete reference cannot pass as one.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
* refactor(ratchet): split attest_refusal into its two questions (DeepSource, #766)
Flags first, checkout second, the same shape select_signature already has, so
adding a rule does not keep pushing one function's branch count up.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
* fix(ratchet): baseline paths are tuples, so a dotted key cannot alias a nested one (Macroscope High, #767)
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
---------
Co-authored-by: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
…roven by executing the step (Cursor, #767 round 1)
The two jq reads that bind the artifact to its run are guarded: under set -e
an unparseable artifact killed the fetch step before the hand-off was written,
so the row never rendered its promised RED-with-reason and Collect died on a
missing args file. The fetch step and the Collect guard are now EXECUTED in
tests under the machine's bash 3.2 with a stubbed gh on PATH: newest success
across pages, bind mismatch, corrupt JSON, missing file, expired artifact,
404 bootstrap, never-succeeded bootstrap, and a missing or empty hand-off into
Collect. Notes name both writer events; the attested notice names the run
instead of a stale initial reason.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
Round 1 (lead's Cursor pass) — all four real, fixed in one commit 9cb86671, test-first. Base is now main (ceeccfdf, #766 merged); attestation #1 on main is green (run 33965753235, artifact published).
medium, unparseable artifact kills the step under set -e:.github/workflows/ratchet.yml:485 — both jq reads are now if ! got_run="$(jq …)" || ! got_sha="$(jq …)", and a failure sets reason="… could not be read as JSON: <jq's first stderr line>" and falls through to the --attestation-unresolved hand-off, exactly like a bind mismatch. Proven by executing the step, not by grepping it:tests/test_ci_ratchet_table.py:2207 runs the real run: block under /bin/bash 3.2 with a stubbed gh and asserts, for {not json, empty, [] and a wrong-typed object: exit 0, an args file, --attestation-unresolved naming run 300. Before the fix that test failed with no args file (the dead-step shape the finding describes).
medium, hand-off only string-checked:tests/test_ci_ratchet_table.py:2112 onward — an executable harness (gh stub serving concatenated --paginate pages and artifacts, sleep stubbed, real jq) pins: newest success across two pages with newer failures first (test_the_fetch_step_takes_the_newest_success_across_every_page), bind mismatch on run id and on sha (…binds_the_artifact_to_the_run_the_api_named), artifact with no file, expired download → unresolved naming workflow_dispatch, HTTP 404 → bootstrap, runs-but-no-success → bootstrap with the count, and the Collect step executed with a missing and with an empty attestation-args.txt (test_collect_fails_before_rendering_when_the_attestation_hand_off_is_missing_or_empty: non-zero, no table).
low, Notes name only push:scripts/ci_ratchet_table.py:853 — "the latest successful push or (no-input) workflow_dispatch run".
low, stale reason in the notice:.github/workflows/ratchet.yml:492 sets it on attested; the notice prints ${reason} only. Pinned by the first executable test (attested — run 300 on main …, and "never read" absent).
Tests: 238 passed. This PR's own table on this head is the live proof of the full path — it should now read matches the main attestation (run 33965753235 · main ceeccfdf…).
The reason will be displayed to describe this comment to others. Learn more.
Explicit in ee4380e: check=False with a comment — the exit code of the executed fetch step is one of the things the test asserts, so raising on non-zero would hide the case under test.
Every Value below was measured by this run. A row this machine cannot measure says n/a — <reason> instead of a number; baselines in Notes name their own machine, method and date and were not measured here.
Row
Status
Value (measured by this run)
Method
Notes
commit provenance
🔴 RED
checkout 7272db4a1d17 is neither 63e388c810c5 nor a merge of it — this run was triggered for a commit it does not have checked out
commit graph + live PR head · in-process · runner
Which commit this whole table is about. On a pull_request event the checkout is GitHub's synthetic merge ref, whose sha is not on the PR — #759's table printed 13fa724278bf while that PR's head was 4632f979 — so this row names the PR-head parent instead, the sha a reviewer can actually see. The comparison sha is read live from repos/{owner}/{repo}/pulls/{n} when the table is collected, not taken from the event payload, because the payload cannot know the run has been overtaken. Residual window, stated rather than papered over: a push landing between that read and the comment being posted is not caught here — the run for that push refreshes the table.
baseline attestation
🟢 GREEN
baseline f421d1a7c5e6 matches the main attestation (run 33979237066 · main 2264b19ea0aa · 2026-09-05T16:53:22Z)
main attestation artifact via Actions API · in-process · runner
What every comparison is measured AGAINST, and who says so. The baseline fields of tests/fixtures/sprint_gate/corpus.json (queries, latency_baseline_ms, thresholds) are compared to the ratchet-attestation artifact of the latest successful push or (no-input) workflow_dispatch run of ratchet-attest.yml on main, fetched through the Actions API — a PR run cannot write to another run's artifacts. A field that differs is RED unless that main run measured the new value; today no runner-side collector measures any baseline field, so today the baseline cannot move by PR at all, and this row says so instead of a hand edit passing. Boundary: the comparator is this PR's checkout of ci_ratchet_table.py, diff-reviewable, not tamper-proof.
provenance
🟢 GREEN
stamped 7272db4a1d17 == HEAD, tree clean
wheel stamp · in-process · runner
Sha half of #749 keg-mode provenance: a keg built from this wheel can answer __build_sha__. The helper-age and served-process predicates need a running BrainBar and are measured only by scripts/sprint_gate.py on an installed Mac. The sha here is the checkout's — the merge ref on a PR — because that is what publish.yml stamps at release time; the PR-head sha this table describes is the one in commit provenance above.
mapped bytes
⚪ n/a
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
socket · installed Mac
Baseline 26.2 GB — installed Mac, socket, 2026-09-03, after R2 drained 15,070 → 0. Up from 16.8 GB because the drain left more vectors mapped under the same cap: the change is the drain, not a leak. Not measured by this run.
search p50/p95
⚪ n/a
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
socket · installed Mac
Margin p50: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Margin p95: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Calibrated on MacBook-Pro.local at 2026-09-01T08:42:22Z under active_sprint_load (tests/fixtures/sprint_gate/corpus.json). Not measured by this run.
idle CPU
⚪ n/a
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
ps sampling · installed Mac
Ceiling: average CPU < 30% over a 60 s window (resource_budget in scripts/sprint_gate.py), ratified and kept as a hard budget. Margin daemon: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Margin helper: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Margin watcher: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Needs the BrainBar daemon, helper and watcher actually running. Not measured by this run.
signature_valid
🟢 GREEN
442 valid / 0 invalid · brainlayer 1.5.14 · macos-15/ARM64 · brew install exited non-zero (outcome: failure); the keg installed and the sweep ran after it
codesign · brew keg · GitHub macOS runner
scripts/release-verify-signatures.sh <keg> codesign-verifies every *.so/*.dylib under libexec/venv. The macOS parity job installs the published tap formula (etanhey/layers/brainlayer), so this row measures the release path — formula, published sdist and Homebrew's relocation — and not this PR's tree. Release-time baseline for the same keg on a different machine: 442 valid / 0 invalid — installed Mac (M4 Max), brew --prefix brainlayer 1.5.11, 2026-09-03.
🟢 GREEN measured, within budget · 🔴 RED measured, out of budget — a finding to clear before merge · ⚪ n/a not measurable on this machine, never guessed.
1 RED row(s) to clear:commit provenance.
Measured on Linux/x86_64 · measured 63e388c810c5 · PR head 63e388c810c5 · checkout 7272db4a1d17 · run · updated 2026-09-05 17:10:23 UTC
…too, and names no direction (Macroscope, #767)
When the attestation is not for this checkout's base, the row no longer says
main "moved" (a sha inequality is not an ordering). It reads the base tip's
committed baseline via git and judges the PR against THAT: a field reverted to
the attested-but-superseded value is a hand edit, RED; a PR carrying what main
has at the base is GREEN with the stale artifact named, not re-applied. The
executable harness declares check=False explicitly where the exit code is the
thing under test (DeepSource).
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
`attested_row` has a cyclomatic complexity of 20 with "high" risk
A function with high cyclomatic complexity can be hard to understand and
maintain. Cyclomatic complexity is a software metric that measures the number of
independent paths through a function. A higher cyclomatic complexity indicates
that the function has more decision points and is more complex.
The reason will be displayed to describe this comment to others. Learn more.
Won't fix in this PR, deliberately: the lead closed review on this head (698400f8, round 2 of 2 ACCEPT) and every path through attested_row — stale base, unreadable base, revert-to-stale, measured key, unchanged, hand edit — is pinned by executed tests, so the split carries no correctness value here and would reopen CI on a PR that merges last in a queue. Follow-up, same shape as attest_refusal's split: lift the "attestation predates the base" branch into its own helper.
…g is not a bootstrap (Cursor, #767 round 2)
Same class as round 1, one layer up: a 200 from `gh api` with a non-JSON body
made the listing jq exit non-zero and killed the step under set -e before any
hand-off was written. Both listing reads are guarded now; garbage is one failed
attempt and three of them are `unresolved`, never bootstrap. Writing the test
found a second shape: an EMPTY 200 body slurps to `[]` and read as "0 runs,
bootstrap" -- the exact fall-back garbage must never reach -- so an empty body
is unreadable too. The git-read base fixture must carry every baseline field,
the shape the writer and reader already enforce. All executed, not grepped.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
Round 2 of 2 (lead's Cursor pass, on 9cb86671) — verified on the current head: the listing jq was still unguarded at ee4380e8. All three real, one commit 698400f8, test-first, executed not grepped.
high, listing jq unguarded under set -e:.github/workflows/ratchet.yml:453 — both listing reads (selected, total) sit inside if [[ ! -s … ]] || ! selected="$(jq …)" || ! total="$(jq …)"; a failure is one failed attempt with reason="… could not be read as JSON in N attempt(s): <jq stderr | empty response body>", and three of them hand over --attestation-unresolved. Writing the test found a second shape you did not name: an EMPTY 200 body — jq -s slurps it to [], which read as "0 run(s) on main … bootstrap", i.e. garbage reaching the one fall-back the row must never take. An empty body is now unreadable too ([[ ! -s "$runs_path" ]]).
medium, no test for corrupt listings:tests/test_ci_ratchet_table.py:2270 executes the real step with the gh stub serving {not json, an HTML 502 page, a truncated paginate page, and an empty body: exit 0, --attestation-unresolved, "3 attempt(s)", and the word bootstrap absent from the output. Against the pre-fix step all four died with no args file (stashed and run, recorded in the report).
low, git_baseline_at accepts an incomplete base:scripts/ci_ratchet_table.py:720 — the base fixture must carry every BASELINE_FIELDS entry, the same shape the writer refuses to attest and the reader refuses to read; test_a_base_fixture_missing_a_baseline_field_is_not_a_reference pins the helper and the bootstrap row going RED on it.
Tests: 245 passed. Live table on this head follows; on ee4380e8 it already rendered GREEN with the non-directional wording doing real work (main had moved to 8aa562c3 — #763 — while the merge ref was still built on ceeccfdf; the old wording would have claimed main moved backwards).
…st-row
Union resolution: the `baseline attestation` row (this branch) and (c)'s margin
store (`--attestations`, `attestations*` Probe fields, margin Notes) coexist.
Two names collided and would have shadowed silently: (c)'s selection class is
now `AttestationStoreSelection`, and the helper-level dotted-key test is
`..._in_the_flattener`. `detect` keeps both selections in their own locals.
Verified: 257 table tests + 59 margin tests, and a local render with five real
main attestations shows the row GREEN against run 33967731131 and the margins
`unmeasured — 0 of the 5` side by side.
Co-Authored-By: brainlayerClaude running claude-fable-5-1 <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Mediumscripts/ci_ratchet_table.py:402
A deeply nested downloaded attestation raises RecursionError from json.loads(), so select_attestation crashes instead of returning an AttestationSelection(problem=...) for the promised RED row. Handle RecursionError alongside the existing parse errors in read_attestation (or contain it here) so malformed input fails closed with a diagnostic.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @scripts/ci_ratchet_table.py around line 402:
A deeply nested downloaded attestation raises `RecursionError` from `json.loads()`, so `select_attestation` crashes instead of returning an `AttestationSelection(problem=...)` for the promised RED row. Handle `RecursionError` alongside the existing parse errors in `read_attestation` (or contain it here) so malformed input fails closed with a diagnostic.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Mediumworkflows/ratchet.yml:457
When 1,000 newer workflow runs fill GitHub's pagination limit without a selected success, this code reports bootstrap even though an older successful attestation may exist. That makes the baseline row fall back to the base commit instead of failing closed as unresolved; treat a full 1,000-run result with no success as unresolved rather than bootstrap.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @.github/workflows/ratchet.yml around line 457:
When 1,000 newer workflow runs fill GitHub's pagination limit without a selected success, this code reports `bootstrap` even though an older successful attestation may exist. That makes the baseline row fall back to the base commit instead of failing closed as `unresolved`; treat a full 1,000-run result with no success as unresolved rather than bootstrap.
…probe
Same DeepSource rule (PTC-W0055) that is currently red on main via #767. My
helper relied on the `check=False` default; it is now explicit, with the reason:
the assert below reports the child's stderr, which is a better failure message
than CalledProcessError's bare exit code.
This does not clear the DeepSource: Python check. That check is red on main's own
tip (7272db4) and was already red on #767's head when #767 merged, carrying two
`subprocess.run` findings in tests/test_ci_ratchet_table.py and a cyclomatic
complexity 20 on attested_row in scripts/ci_ratchet_table.py. None of those three
files is in this PR's diff.
Co-Authored-By: brainlayerClaude running claude-opus-5 <noreply@anthropic.com>
…s sklearn import behind it (#782)
* perf(hook): cap and skip UserPromptSubmit injection, and fix the 790ms sklearn import behind it
W18. The brief proposed a 600-char cap, a `^/` skip, and a <12-word skip. Measured
against 5,360 real hook fires over 30 days first: the cap saves 0.3%, the `^/` rule
never fires (slash commands arrive as `<command-name>` envelopes), and a raw 12-word
skip kills real questions. So:
- Relay/envelope skip — `[report]`, cross-session messages, `<command-name>` and
`<local-command-*>` envelopes: 12.0% of fires, 10.0% of injected chars, no
retrieval risk.
- Short-prompt skip narrowed to <12 words AND <=1 surviving keyword AND not on
follow_up/entity_lookup/hebrew_query: 4.9% of fires, 4.8% of chars. Corrections
still get their notice — short prompts are where corrections live.
- 600-char cap implemented as a bound against regression, cutting at a line
boundary with a pointer, never below one line.
The real cost was elsewhere. `from brainlayer.pipeline.correction_detection import
detect_correction` at module scope pulled pipeline/__init__ -> semantic_style ->
sklearn: 790ms of the hook's 810ms floor, paid on every prompt including skipped
ones. semantic_style already defers sentence_transformers via find_spec for exactly
this reason; sklearn was missed. Deferred it the same way, and made the hook's
detect_correction lazy.
Measured on 20 real prompts from the lead's transcript: 3,973 -> 2,018 injected
chars (-49.2%), 880 -> 125 ms per prompt (-85.8%). No long human prompt lost a char.
Tests: 36 new in tests/test_prompt_search_cap.py, including two import-cost guards
verified to fail when each change is reverted. Full unit suite 4,603 passed.
Co-Authored-By: brainlayerClaude running claude-opus-5 <noreply@anthropic.com>
* fix(hook): derive the cap's note reserve instead of hand-setting it
Self-review caught a real off-by-one in the cap I had just written. The reserve
was 56, but the pointer line is 56 chars at one digit plus the newline that joins
it -- 57. A budget filled to the byte therefore produced 602 chars against a 600
cap:
lines = ["x" * 44] + ["y" * 49] * 30 -> 602
The reserve is now derived from the note template itself (`len(...format(9999)) + 1`)
so it cannot drift from the string again. Two tests pin it: the exact 602 case, and
the invariant across 2/3/7/15/40/120 lines. A 200,000-trial fuzz over random line
counts and widths reports 0 violations with a worst-case total of exactly 600.
Co-Authored-By: brainlayerClaude running claude-opus-5 <noreply@anthropic.com>
* fix(hook): address review round 1 — two of the four findings were real bugs I shipped
Macroscope reviewed and was right four times.
1. The `entity_lookup` exemption was DEAD CODE. The short-prompt gate ran where
`classify_prompt(prompt)` had not yet seen any entities, so it could never
return `entity_lookup` -- a one-word entity question ("Etan?") was skipped
instead of answered. Gate moved to after entity reclassification. A test
asserts the source ORDER, not just the predicate.
2. Relay prefixes were matched with `startswith` alone, so a prompt that opened
by quoting `[report]` or `<command-name` while asking about it got skipped.
Each shape must now carry its envelope structure: a closing tag, a `from=`
attribute, or the path a report ping names.
3. The deferred import was eating the search deadline. `start` is captured
inside main(), so moving the ~105ms pipeline import out of module scope
charged it against DEADLINE_MS=450, where a slow first import could silently
suppress entity detection and FTS -- the one thing the deadline must not do.
`search_elapsed_ms()` subtracts deferred-import time; `elapsed_ms()` still
reports true wall time to telemetry.
4. Capped results were still registered as injected. Chunk IDs the cap withheld
went into the session dedup file anyway, so later prompts would suppress
chunks the agent never received -- silent memory loss. `cap_injection()` now
returns `(kept, dropped)` and registration is trimmed to the survivors.
DeepSource: both `global` statements removed (deferred-import state moved into a
dict) and the 20 pytest methods that ignore `self` are now `@staticmethod`.
Honest cost of being right: the 30-day projection falls from 15.1% to 13.5% of
injected chars (~65K -> ~58K tokens/30d). 95 short prompts keep their entity
context and 6 quoted-prefix prompts are no longer treated as relays. Correctness
bought with savings, in that direction.
Tests 36 -> 63. Full suite 4,884 passed after merging main.
Co-Authored-By: brainlayerClaude running claude-opus-5 <noreply@anthropic.com>
* test(hook): make subprocess.run's check= explicit in the import-cost probe
Same DeepSource rule (PTC-W0055) that is currently red on main via #767. My
helper relied on the `check=False` default; it is now explicit, with the reason:
the assert below reports the child's stderr, which is a better failure message
than CalledProcessError's bare exit code.
This does not clear the DeepSource: Python check. That check is red on main's own
tip (7272db4) and was already red on #767's head when #767 merged, carrying two
`subprocess.run` findings in tests/test_ci_ratchet_table.py and a cyclomatic
complexity 20 on attested_row in scripts/ci_ratchet_table.py. None of those three
files is in this PR's diff.
Co-Authored-By: brainlayerClaude running claude-opus-5 <noreply@anthropic.com>
---------
Co-authored-by: brainlayerClaude running claude-opus-5 <noreply@anthropic.com>
EtanHey
added
the
size:L
Tight-loop PR size: over 400 hand-written lines changed; canon 9 needs a one-line why
label
Sep 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SSmall changesize:LTight-loop PR size: over 400 hand-written lines changed; canon 9 needs a one-line why
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
![deepsource-io[bot]](https://avatars.githubusercontent.com/in/16372?s=60&v=4){kind=small}
Summary — ratchet bolt (b), slice 2 of 2: the row that makes the baseline unforgeable
Stacked on #766 (base branch
wt/ratchet-b1-attest-store). Slice 1 built the store; this is the consumer. Merge #766 first, then retarget this PR tomainbefore deleting that branch.ratchet.ymlgainsactions: readand a step, Fetch the baseline attestation from main, that listsratchet-attest.yml's runs onmainthrough the Actions API, picks the newest successfulpush/workflow_dispatchrun, downloads itsratchet-attestationartifact, and binds it to the run the API named — the artifact'srun_idandmain_shamust both match, or the hand-off isunresolved. Three retries on each API read, like the live-head read. Four outcomes, none silent:attested,bootstrap,unresolved; and an empty hand-off makes the collect step refuse to render.baseline attestation, right aftercommit provenance("which commit, then what it is measured against, then the numbers"). It diffs the PR's baseline fields (queries,latency_baseline_ms,thresholds) against the attestation, field by field:old → new;measured[path]) → 🟢 "moved to values measured by main". That is the legitimate path, and the only one.HTTP 404— the writer is not onmainyet) is the one case that is neither a measurement nor a finding. The row then compares against the base tip's committed fixture viagit show, merge-ref only — a direct checkout's first parent is the PR's own branch and is refused — and says it is bootstrapping. Once one success exists on main this branch is never taken again: an expired artifact is 🔴 unresolved, refreshed by oneworkflow_dispatch.The boundary, stated exactly (per #761's reviewer)
No INPUT to this row comes from the PR tree: the run list, the artifact and its binding are GitHub's. The comparator is this PR's checkout of
ci_ratchet_table.pyand could be rewritten to answer GREEN. Diff-reviewable, not tamper-proof. The docstring, the Notes column and the workflow comment say this and nothing wider.Premise check — two corrections to the brief
mapped bytes,search p50/p95,idle CPUare fixture-boundn/aon every hosted runner;signature_validcompares counts to zero. The baseline is read byscripts/sprint_gate.pyon the installed Mac and quoted in the table's Notes. So the harm was real (the installed gate and the reader-facing claim) but it was never a CI RED/GREEN flip.measuredis{}on every attestation today — no runner-side collector measures a baseline field. Consequence, stated in the row's Notes: today the baseline cannot move by PR at all. The key to this lock is a collector (a self-hosted installed-Mac runner, or a CI-measured count promoted into the baseline), not a hand — which is the ratchet's own rule. The legitimate path exists in code and is tested end to end through the CLI; it has nothing to carry until something is measured. Etan's call whether that posture is intended or a human-ratified override should exist.Definition of done — proven against the old sources
Swapped
3ee7c279'sscripts/ci_ratchet_table.pyin and ran the four bolt tests:Same four on this branch:
4 passed. Full file:205 passed(plus slice 1's reader tests, kept).The fetch step's bash was run locally against the live API (
REPO=EtanHey/brainlayer) and producedbootstrap — ratchet-attest.yml is not a registered workflow on main yet (HTTP 404), which is what this PR's own table should render. The live row on this PR is the acceptance test; the report's DONE marker waits for it.Size
S — 358 non-test lines (
ratchet.yml+102, collector +256); tests +429.Test plan
ruff checkandruff format --checkonscripts/andtests/tests/test_ci_ratchet_table.py: 205 passed (row-level, real-git bootstrap on a merge ref, malformed artifacts, end-to-end CLI, pyyaml assertions on the fetch step —!cancelled()neveralways(), exactly one hand-off flag, retries, no reader of the checkout,actions: readand never write)tests/test_kg_judge.pygit-shelling registration: passes (_clean_git_env()on the new merge-ref repo helper)baseline attestation🟢 bootstrap andcommit provenance🟢 on the final headmatches the main attestation (run N · main <sha>)— brainlayerClaude (worker) · claude-code/claude-fable-5-1
Note
Medium Risk
Changes CI gate semantics for baseline fixture edits and adds Actions API dependency on PR runs; mis-fetch or comparator bugs could block merges or miss hand edits, but scope is isolated to ratchet workflow/collector with extensive fail-closed tests.
Overview
Adds baseline attestation as the second ratchet row so sprint-gate baseline fields (
queries,latency_baseline_ms,thresholds) are checked against a reference PRs cannot forge.Workflow (
ratchet.yml): Grants read-onlyactionspermission and a new step that paginates successfulratchet-attest.ymlruns onmain, downloadsratchet-attestation, and verifiesrun_id/main_shamatch the artifact. Outcomes hand off exactly one of--attestation,--attestation-bootstrap, or--attestation-unresolved(with retries and guardedjqso corrupt API bodies fail closed). Collect refuses to run without a non-empty attestation hand-off.Collector (
ci_ratchet_table.py): Compares the checked-out corpus baseline to the attested baseline; unlicensed edits are RED with an explicit hand-edit message, changes matching main’smeasuredmap (type-sensitive) are GREEN, and pre-first-success on main uses merge-basegit showbootstrap only. Stale attestations (artifact sha ≠ base tip) also diff against the base commit so reverting to an old attested value cannot pass silently.Tests: Broad coverage for row logic, CLI exit codes, workflow YAML contracts, and executed bash for the fetch/collect steps.
Reviewed by Cursor Bugbot for commit 63e388c. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add baseline-attestation row to ratchet table so gate compares against main-measured values
ratchet-attest.ymlartifact frommain, verifies its run ID and SHA, and hands the artifact (or bootstrap/unresolved state) to the Python collector instead of letting the PR supply its own baseline.baseline attestationrow that reports GREEN when the PR baseline is unchanged or matches a value main measured, and RED for hand-edited fields, type-coerced values, or stale/reverted attestations. Bootstrap mode compares against the merge-ref base commit's baseline before any main attestation exists.licensedhelper now requires exact value and type match for a baseline change to count as main-measured, preventing boolean/count coercion and dotted-key aliasing.select_attestationfails closed for conflicting--attestationmodes, malformed artifacts, and unresolved fetches; a missing or empty attestation hand-off now fails collection before rendering a table instead of renderingn/a.Macroscope summarized 63e388c.