Give the packer a curve dimension, so #139 is a one-line change and not a project - #190
Open
defenwycke wants to merge 1 commit into
Open
Give the packer a curve dimension, so #139 is a one-line change and not a project#190defenwycke wants to merge 1 commit into
defenwycke wants to merge 1 commit into
Conversation
…ot a project `predicted_ec_ops` counted signature verifications without distinguishing ECDSA from Schnorr, because today they cost the same. After #139 they do not: ECDSA drops to ~191K cycles while Schnorr keeps running libsecp's BIP340 code at 1,950,000 -- a 10.2x divergence the packer cannot see. That is not cosmetic. A block's wall-clock is its SLOWEST chunk, so simulated on block 962,000 -- which is only 2.7% Schnorr, the mildest case available: before #139 900M cycles 1.00x after, packer unchanged 305M cycles 2.95x straggler 2.45x after, packer type-aware 130M cycles 6.95x straggler 1.04x Refitting is worth 2.36x. It is a PREREQUISITE for #139, not a follow-up: without it the win is 2.95x and the fidelity is spent either way. `predicted_sig_ops` now returns `SigOps { ecdsa, schnorr }`. Taproot is BIP340 on BOTH paths -- key path and tapscript -- and everything else is ECDSA, matching the classifier `pack_after_139.py` already uses. `predicted_ec_ops` remains as a total-returning wrapper for the profile display, which reports a count. DELIBERATELY A NO-OP TODAY. COST_PER_SCHNORR_OP is introduced equal to COST_PER_EC_OP, so costs, partitions and chunks are unchanged. Verified rather than argued: chunk-profile output captured from this binary and from a rebuilt main, 526 lines across three blocks (962000, 741000, 140000) at N = 4, 16, 64 -- byte-identical. #139 then becomes a one-line edit to COST_PER_EC_OP, and the packer work no longer waits on the fidelity decision. Four tests, one of which exists because the others are not enough: - taproot classifies as Schnorr, the rest as ECDSA - cost is curve-blind while the coefficients are equal (the no-op property), and it asserts the constants ARE equal, so it fails loudly when #139 lands rather than silently protecting a premise that has gone - ⚠ pricing the curves apart actually CHANGES the cost. A no-op test alone would pass for a refit that does nothing whatsoever; this shows the machinery bites - an unparseable input is charged one ECDSA verify and never panics, since this is only a scheduling hint `cost_with(ec, schnorr)` is split out so that third test can price the curves apart without editing a constant. This is the fourth time a stale packer coefficient has done measurable harm -- the byte term went 182 -> 36 -> 6 across #136 and #137. Unlike those, this one could not be fixed by refitting a number: the model had no term for input type at all.
defenwycke
added a commit
that referenced
this pull request
Aug 27, 2026
…nnot fail `FLEET_SIZING.md` §4 lists "14.34 G is modelled" as reason #3 not to trust the card count, and every figure in the sixteen-card plan divides by it. Measured now, on a laptop with no GPU and no chunk receipts: **14.057 G**, so the model was +2.0% high. Chunk work on one L40S is 14,367 s rather than 14,656, the one-card total is 15,933 s, and the bar for sixteen cards moves 1.95x to 1.92x. The conclusion is unchanged, which is the useful part -- the denominator now rests on a measurement. Two things fell out of the same run. Cost-packing is worth **1.18x on the slowest chunk** -- which is what a block's wall-clock actually is -- for a total-cycles cost of +0.06%. That trade was argued from a model; it is measured now, and `reproduce/METHOD_ID`'s "slightly MORE" is quantified. And the packer's own balance metric is computed from PREDICTIONS. It prints 1.00x for a partition that measures 1.059x, so it reports perfect balance whatever the guest does. The per-chunk error is signed rather than noisy and it tracks payload size: the three highest-byte chunks measure -12.3%, -12.7% and -6.2% against prediction, so the byte term is still over-charging after #136 and #137 -- the fourth time a stale packer coefficient has done measurable harm. Neither is a lever on the sixteen-card scale, but #190 is adding a curve dimension to this packer and the refit is a #139 prerequisite worth 2.36x, so these 32 measured points are what it should be fitted against.
defenwycke
added a commit
that referenced
this pull request
Aug 27, 2026
…is per-input Workstream B, following the observation that this project's three largest wins were all in the plumbing and none in consensus code. `write_chunk_inputs` coalesces consecutive inputs sharing a `tx_idx` and writes each transaction and prevout blob ONCE per group. `input_costs` prices the same work for EVERY input at full transaction size. Summed over block 962,000 the per-input charge counts 53.40 MB where each transaction counted once is 1.53 MB -- 34.9x apart -- and the measured consequence is already on record: the three highest-byte chunks come in 12.3%, 12.7% and 6.2% under prediction, and the predicted straggler reads 1.001x against a measured 1.059x. It would be wrong to call this a stale constant, and I nearly did. `git log -S` puts the last change to `COST_PER_INPUT_BYTE = 6` in the same commit that introduced grouping. The value was chosen with grouping in view. The defect is a missing dimension. The byte term carries marshalling, which grouping made per-GROUP, and `input_bind`'s re-hash, which is still per-INPUT, and one constant cannot express both scalings -- the comment above it already says ~2.5 of the 6 is the re-hash. So the fix is a second term rather than a refit, and it is the same shape of defect as the missing type dimension #190 is adding a curve dimension for. Teach the packer both at once. The re-hash itself prices at ~133 M cycles, 0.95% of the block -- small, though 87% of those bytes sit in 10% of the inputs, so it lands on the straggler rather than spreading. One thing to stop quoting until it is re-derived: `pack_after_139.py` amortises the byte term across a transaction's inputs while the real packer charges it in full per input. That script is where the 2.36x packer-refit figure comes from.
defenwycke
added a commit
that referenced
this pull request
Aug 28, 2026
#139 measured at 8.00x (middle path) and 9.10x (wholesale) on GPU proving wall, and the aggregate measured at 1.81x on two workers. Together those take a near-tip block from 32 cards to about 7-9. Both caveats are stated where the number is, not further down. The aggregate saturates at N=2 on a four-chunk test and nobody yet knows whether that is join-tree width, which is harmless at sixteen chunks, or a coordinator bottleneck, which would cap the aggregate near 1.8x and put it at 865 s — above the whole budget at any fleet size. And #190 has to land or the post-#139 straggler goes to 2.45x. Claude-Session: https://claude.ai/code/session_015nyynoX7kXHe7zCF3j6Jbf
defenwycke
added a commit
that referenced
this pull request
Aug 29, 2026
The packer branch shipped COST_PER_EC_OP equal to COST_PER_SCHNORR_OP on purpose, so adding a curve dimension was a provable no-op -- same costs, same partition, same chunks. I launched an arm without making the one-line edit its own comment describes, and it reproduced arm S almost exactly: 240/218/168/184 against 239/221/171/181. It was measuring nothing. Setting ECDSA to its post-#139 cost makes the two curves diverge 13.8x, which is what lets the packer balance chunks by what they will ACTUALLY cost once the accelerator is on. The measurement that motivates it: 16/16 paired chunks on one L40S at po2 22, per-chunk speedup 2.046x to 4.813x, correlating with taproot share at -0.875. The packer predicts all 16 chunks within 1.00x while the real straggler goes 1.054 -> 2.118. Wall-clock is the slowest chunk, so that is ~7 cards. Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL
defenwycke
added a commit
that referenced
this pull request
Aug 29, 2026
Arm A exported HAZYNC_LIFTX_ACCEL=1 but build_arm only ever applied patch 0005, so hazync_lift_x would have compiled and never been called: the arm would have measured a no-op and reported that G1 does not help. That is the third instance of this shape in one day -- #139 without patch 0005 builds stock libsecp, #190 without its constant edit repacks nothing -- so this also greps group_impl.h for the macro afterwards rather than trusting patch's exit code. Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL
defenwycke
added a commit
that referenced
this pull request
Aug 29, 2026
patches/0007 adds an #ifdef HAZYNC_LIFTX_ACCEL block to group_impl.h, and prover/methods/guest/build.rs is what has to .define() that macro for the C compile. I wired the Rust feature in methods/build.rs and never added the define, so the patch applied, METHOD_ID moved -- group_impl.h is guest C source, so it moves either way -- and the guest compiled libsecp's own fe_sqrt. The build was green and the lever was off. That is the fourth instance of this exact shape today: #139 without patch 0005, #190 without its constant edit, arm A without patch 0007 applied at all, and now this. So stop relying on catching it by eye. The arm now greps the binary for the entry points it claims to use, with a negative control that must NOT match -- a check with no failing case is not a check. My first attempt at this check was itself broken: swallowed the zero case and printed nothing, which read as success. Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
predicted_ec_opscounted signature verifications without distinguishing ECDSA from Schnorr,because today they cost the same. After #139 they don't: ECDSA drops to ~191K cycles while
Schnorr keeps running libsecp's BIP340 code at 1,950,000 — a 10.2× divergence the packer
cannot see.
That isn't cosmetic. A block's wall-clock is its slowest chunk. Simulated on block 962,000,
which is only 2.7% Schnorr — the mildest case available:
Refitting is worth 2.36×, and it's a prerequisite rather than a follow-up — without it #139
delivers 2.95× instead of 6.95×, and the fidelity is spent either way.
The change
predicted_sig_opsreturnsSigOps { ecdsa, schnorr }. Taproot is BIP340 on both paths — keypath and tapscript — everything else is ECDSA, matching the classifier
pack_after_139.pyalready uses.
predicted_ec_opsstays as a total-returning wrapper for the profile display,which reports a count rather than a cost.
Deliberately a no-op today — verified, not argued
COST_PER_SCHNORR_OPis introduced equal toCOST_PER_EC_OP, so costs, partitions and chunksare unchanged. Confirmed empirically:
chunk-profileoutput captured from this binary and from arebuilt
main, 526 lines across three blocks (962000, 741000, 140000) at N = 4, 16, 64 —byte-identical.
So this lands before the #139 fidelity decision rather than after it, and #139 becomes a
one-line edit to
COST_PER_EC_OP.Four tests, one of which exists because the others aren't enough
equal, so it fails loudly when bigint2 verifies ECDSA 13.78x faster than libsecp256k1 in-guest — decide what to do about it #139 lands rather than silently protecting a premise that has
gone
refit that does nothing whatsoever. This one shows the machinery bites.
hint, and a wrong guess must never be able to kill a prover run
cost_with(ec, schnorr)is split out so that third test can price the curves apart withoutediting a constant.
Context
This is the fourth time a stale packer coefficient has done measurable harm — the byte term
went 182 → 36 → 6 across #136 and #137, each stale value optimising against work that was no
longer there. Unlike those, this one couldn't be fixed by refitting a number: the model had no
term for input type at all.
Host-only. No
METHOD_ID, no guest change, no fidelity cost.