K=1 rowwise pilot machinery: clone the spine, assign the ladder, measure per-area support (#761) - #815
K=1 rowwise pilot machinery: clone the spine, assign the ladder, measure per-area support (#761)#815juaristi22 wants to merge 11 commits into
Conversation
The rowwise drivers' pipeline names move from uk-locals-* to uk-local-* so the derived chain scope is uk/local (Maria's naming ruling on #761), and the scope is ratified for the local-areas line's first archived run: the scope_declared migration, the DECLARED_SCOPES mirror, and the README vocabulary move together, per the closed-world convention. No archived rows existed under the old names (the scope was never ratified), so the rename is history-free. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The household_id-mod-10^8 rule was written for the certified pool's
tier-offset id scheme. The microcosm spine's ids stack flag-driven
offsets (sernum + 1{spi}.10^d + 1{cgt_clone}.10^(d+1) +
1{band_donor}.10^(d+2)) and carry explicit lineage columns, so
apply_uk_source_lineage_modulus now refuses any input carrying them --
the columns are the source of truth, and a derived modulus layer would
mint a false second lineage. The dry-run plan and build manifest gain an
additive explicit lineage block (distinct source households, per-channel
splits, flag counts, computed at pre-clone grain), null for pool inputs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
assign_uk_geography_ladder silently overwrites oa/lsoa/msoa codes and would leave stale *_oa columns riding through beside its own outputs. The ladder clone path and its dry-run plan now refuse a household input that already carries any ladder geography column or legacy *_oa name (region, the required enum input, excepted) -- two disagreeing constituency columns on one artifact is exactly the laundering the fences exist to prevent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pilot's contract is a sha-pinned spine and a sha-pinned ladder. Both pins follow the calibration seam's convention (measured-vs-pinned refusal before any bytes are parsed or output side effect happens), and each artifact block records pin_verified. The ladder block additionally cross-checks the measured sha against the committed local_area_crosswalk.json ladder_artifact_sha256 pin -- informational, never a refusal, so synthetic test ladders keep working while a real run records whether it matched the ratified artifact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
expected_uk_ladder_area_support computes exact expected rows per constituency (household-count share within region x input region mix x n_clones) and per local authority (OA population shares within constituency), sharing the sampler's own stage-weight helpers so the expectation cannot drift from the draw. The ladder dry-run plan gains an expected_support block beside realized_support, zero-support areas included. The stage-two zero-mass refusal now names the constituency instead of surfacing as an rng.choice probability error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
uk_area_support_summary is the frame-agnostic core (one groupby instead of the per-area scan, numerically identical); the matrix-coupled rowwise_area_support_summary becomes a thin wrapper with its signature untouched. The new uk_ladder_area_support_summary computes the same per-area ESS / distinct-source table for a plain ladder-assigned frame on the full ladder roster, and uk_region_mix reports the input's region composition as shares. The ladder route's manifest and dry-run plan gain area_support (min/median rows, ESS, distinct sources, bottom-15 by rows and by ESS, with the source_basis recorded) and region_mix blocks, and the real build writes the full table as area_support_summary.csv. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--candidate-clone-counts (dry-run only, refused on a real build before any side effect) plans every candidate K independently at the build seed: per-K rows and byte estimates on both routes, realized and analytic expected support, and on the ladder route the per-area support stats. The fenced ladder plan body is extracted into _ladder_planned_assignment so the primary plan and every candidate K run the identical clone/assign/gate pipeline. The crosswalk route's candidate blocks omit area_support (no ladder roster) and say so in their basis string. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One attempt, disposition iterating: uk-local-rowwise ladder route, f100, seed 42, spine-i (0cce0399) through the ratified ladder (9c6d56b9) -- gate passed 650/650, zero missing-geography rows, mass conserved exactly. Row digest a4d2c0b2. The chain opens at this row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Measured on spine-i: the multi-K plan (K in {1,2,4,10}) and the real K=1
build. 650/650 constituencies at every K; constituency min-ESS climbs
15.3 -> 28.8 -> 86.4 -> 245; the analytic expectation tracks the
realized draw on both grains; the K=1 build conserves mass exactly at
1.51 GB peak RSS. Recommendation: K=4 for the first calibrated
candidate; adjudication pending Round B (spine-j) per Maria's ruling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
test_committed_archives_are_scoped_by_country pins the committed archive inventory; archiving the genesis uk/local row legitimately extends it. Caught by CI's engine-free lanes -- the local runs predated the archive commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Automated review pass (Claude Code, high effort, diff only — no execution; test hunks skipped). Three findings. 1.
|
Closes #761 (#495 increment 5, workstream G of #665).
The K=1 rowwise pilot machinery: clone the microcosm-built spine, assign geography through the
ratified OA ladder, and measure per-area support — no solve, no binding, no publication. The epic's
K table was sized against the retired 535,080-row certified compact; the spine that replaced it is
52,846 households, so K re-adjudicates from measured support. This PR lands the measurement
machinery; the licensed runs follow on this branch (two rounds per María's ruling: spine-i now as
shakedown + control, spine-j for the K ruling once #750's licensed phase lands).
What lands
uk-local-*and itsuk/localLogbook scope is ratified — migration +DECLARED_SCOPESmirror + README move together (María's naming ruling: singularuk/local)household_id mod 10^8lineage rule restates for the spine: the pool-only modulus is refused on inputs carrying explicit lineage columns, and the plan/manifest gain an additiveexplicitlineage block (distinct sources, per-channel splits, flag counts, pre-clone grain)*_oacolumns cannot ride through beside the ladder's own outputs--input-sha256/--ladder-sha256pins (calibration-seam convention: refuse before any bytes parse or side effect), with an informational cross-check against the committedlocal_area_crosswalk.jsonladder pinexpected_uk_ladder_area_support— the exact analytic expectation (constituency household-count share within region × input region mix × K; OA population shares for LA), sharing the sampler's own stage-weight helpers so it cannot drift from the drawuk_area_support_summary(one groupby, numerically identical), matrix-freeuk_ladder_area_support_summary,uk_region_mix; ladder manifest + dry-run gainarea_support/region_mixblocks and the real build writesarea_support_summary.csv--candidate-clone-countsplans rows/bytes/support at every candidate K in one dry-run (dry-run-only surface, refused on a real build); every candidate K runs the identical extracted clone/assign/gate pipelineWhat does not change
No solve and no binding:
build_uk_rowwise_candidate.pyuntouched beyond the two-line pipelinerename,
local_doctrine.pybyte-identical, fences and adjudication registers untouched. TheLogbook rung stays
f100— rungs are sample-fraction tokens (#624 grammar) and the pilot consumes100% of the spine at any K. Dry-runs still append no Logbook row. Weight fences (kind carriage,
mass-log currency, exact conservation at 1e-9, the single clone mass record, the write-path
re-gate) already did what #761 items 3–4 require and are untouched.
Still to land on this branch
Round A (spine-i,LANDED: the multi-K dry-run plan and the real K=1 build0cce0399…)both ran on the licensed machine — ladder gate passed 650/650 constituencies and 361/361 LAs,
zero missing-geography rows, mass conserved exactly (abs_delta 0.0), IMPORTANCE kind carried
with one appended mass record, 16,288 distinct sources via the explicit lineage columns, 17 s /
1.51 GB peak RSS. Constituency min-ESS by K: 15.3 (K=1) → 28.8 → 86.4 → 245 (K=10). The genesis
uk/localLogbook row is archived and the chain validates. Receipts:experiments/761-uk-rowwise-pilot-receipts.md(+ per-K stats JSON); recommendation there isK=4 for First calibrated rowwise UK local candidate (#495 increment 6) #762, adjudication pending Round B.
ruling cites spine-j with spine-i as control.
(clone-count-as-declaration + measured pre/post-clone grain) posts from the receipts.
Note for the reviewer: the hosted-store migration (
20260829000000_logbook_uk_local_scope.sql)needs manual application; until then the licensed run's live insert warns and continues — local
spooling and the git export path are unaffected.
Verification
Ruff clean;
ci_test_groups.py --verifyok; the fullukgroup ran green pre-rebase (the twotest_uk_local_area_crosswalkfailures were the moved #795 branch, resolved by rebasing onto themerged main); post-rebase the delta surface (16 files: logbook, rowwise, ladder, local-rowwise,
crosswalk, census, spec-pin suites) is green. PR CI is the authoritative full run.
🤖 Generated with Claude Code