Skip to content

fix(storage): drop the five retired share-batch-chain tables (#585) - #828

Merged
defenwycke merged 1 commit into
mainfrom
fix/585-drop-retired-sbc-tables
Sep 2, 2026
Merged

fix(storage): drop the five retired share-batch-chain tables (#585)#828
defenwycke merged 1 commit into
mainfrom
fix/585-drop-retired-sbc-tables

Conversation

@defenwycke

Copy link
Copy Markdown
Contributor

First substantive step on #585 after the backup pruning in #826.

What this reclaims

SBC was deleted in #703, the share_batch_shadow flag is gone from config, and #692 closed on 2026-08-22. The data outlived the feature. Measured on ghost-vm5 today:

table rows size
sbc_batches 1390 133 MB
sbc_certs 1389 2 MB
sbc_watermarks 6 ~0
sbc_balances 5 ~0
sbc_quarantine 1 ~0

~135 MB per node, ~1.1 GB fleet-wide. sbc_batches dominates because each row stored verbatim JSON — roughly 95 KB apiece.

Verified first: outside migrations.rs, the only mentions of any of these tables are two doc comments in shard_store.rs comparing the shard's design to SBC's. No read path, no write path, no flag to re-enable it.

Why this doesn't use v57's "must be empty" guard

migrate_v57 refuses to drop a non-empty wraith_bonds because every row was escrowed value — dropping it would silently release or strand sats. That reasoning is about money, not a blanket rule. These rows are retired accounting history for a mechanism the shard replaced.

⚠ The bug a test caught in my own verification

The migration verifies its work rather than assuming it. My first version read:

SELECT COUNT(*) FROM sqlite_master WHERE name LIKE 'sbc_%'

That is blind to what it claims to check — the indexes are idx_sbc_* and sqlite_autoindex_sbc_*, so a name-prefix match catches none of them, and the check would report success with orphaned indexes still in the schema. It now also matches on tbl_name (the table an index belongs to) and escapes the _ wildcard so it asserts the real prefix.

Control Result
Leave an index undropped, current check test fails
Leave an index undropped, old name-only check test passes ← the bug

Two existing tests inverted, not deleted

v50_stores_balances_... and v53_applies_on_v52... both asserted these tables were present at head. Both are flipped to assert they're gone, so a future migration recreating them has to argue for it. The v50 replacement carries a control asserting the shard tables that replaced SBC do exist — otherwise it would pass just as happily against a database that failed to migrate at all.

⚠ Frees pages, does not shrink the file

auto_vacuum=0 and the freelist is essentially empty, so the space is reused internally and growth slows — the file does not get smaller. Returning it to the OS needs VACUUM, which wants twice the DB free and took ghost-vm6 down once. Deliberately not done here; tracked separately on #585, where it is now affordable (23–25 GB free vs a 3.7 GB DB).

cargo test -p ghost-storage --lib: 234 passed, 0 failed. Clippy and fmt clean.

Deploy note

Bumps SCHEMA_VERSION to 59, so a migration runs on each node at next roll. v58 took ~81s and left a WAL high-water mark; this is a set of DROP TABLEs rather than a column rewrite, so it should be cheaper, but expect the same WAL behaviour.

https://claude.ai/code/session_01Td1vvfowptTTnu88qG2iym

SBC was deleted in #703, `share_batch_shadow` was turned off fleet-wide and
the flag removed from config, and #692 closed on 2026-08-22. What survived the
deletion was the DATA. Migrations v50/v51/v52 created these tables, nothing has
read or written them since, and every backup and page-cache read has carried
them ever since.

Measured on ghost-vm5, 2026-09-02:

    sbc_batches      1390 rows   133 MB
    sbc_certs        1389 rows     2 MB
    sbc_watermarks      6 rows    ~0 MB
    sbc_balances        5 rows    ~0 MB
    sbc_quarantine      1 row     ~0 MB

~135 MB per node, roughly 1.1 GB across the fleet. `sbc_batches` dominates
because each row held a verbatim JSON payload, ~95 KB apiece.

Verified before writing it: outside `migrations.rs`, the only mentions of any
of these tables are two doc comments in `shard_store.rs` comparing the shard's
design to SBC's. No read path, no write path, no flag left to turn it back on.

Unlike v57 this does NOT refuse a non-empty table. That guard exists on
`wraith_bonds` because every row was escrowed VALUE and dropping it would
release or strand sats. These rows are retired accounting history for a
mechanism the shard replaced.

⚠ The verification step originally read `WHERE name LIKE 'sbc_%'`, and a test
caught that it was blind to exactly what it claimed to check: the indexes are
called `idx_sbc_*` and `sqlite_autoindex_sbc_*`, so a name-prefix test matches
none of them and would report success with orphaned indexes still in the
schema. It now checks `tbl_name` as well, and escapes the `_` wildcard so it
asserts the real prefix rather than something close to it. Control: with the
old name-only check and a deliberately undropped index, the test PASSES; with
the current check it fails.

Two existing tests asserted these tables were PRESENT at head — v50's schema
guard and v53's "additive, leaves sbc alone" check. Both are inverted rather
than deleted: at head the tables must be gone, and a future migration
recreating them should have to argue for it. v50's replacement carries a
control asserting the shard tables that replaced it DO exist, so it cannot
pass against a database that simply failed to migrate.

⚠ This frees pages; it does NOT shrink the file. `auto_vacuum=0` and the
freelist is essentially empty, so the space is reused internally and growth
slows rather than the file getting smaller. Returning it to the OS needs
`VACUUM`, which needs twice the database free and took ghost-vm6 down once —
deliberately not done here and tracked separately on #585.

Claude-Session: https://claude.ai/code/session_01Td1vvfowptTTnu88qG2iym
@defenwycke
defenwycke merged commit 9e2447b into main Sep 2, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant