Skip to content

The CUDA default is po2 21, not 22 — and it costs ~11% - #198

Open
defenwycke wants to merge 1 commit into
mainfrom
docs/fix-seg-po2-default
Open

The CUDA default is po2 21, not 22 — and it costs ~11%#198
defenwycke wants to merge 1 commit into
mainfrom
docs/fix-seg-po2-default

Conversation

@defenwycke

Copy link
Copy Markdown
Contributor

SEGMENT_DISTRIBUTION.md said "22 is the default on CUDA and peaks at ~40.6 GB of VRAM". Both halves are wrong. main.rs is unambiguous:

fn seg_po2() -> u32 {
    std::env::var("HAZYNC_SEG_PO2").ok().and_then(|s| s.parse().ok())
        .unwrap_or(if cfg!(feature = "cuda") { 21 } else { 20 })
}

po2 21 peaks at ~22 GB — measured today at 22,521 and 22,009 MiB on an L40S. That is half the 40.6 GB the docs implied, and 40.6 GB is what po2 22 costs, which you only get by setting the variable explicitly.

Why it matters

1. It silently split the project's measurements. Any run that did not set HAZYNC_SEG_PO2 was at po2 21. Figures labelled "at po2 22" elsewhere were only that if someone set it.

2. A fleet's VRAM headroom was being sized against a number twice the real default.

3. The default leaves ~11% on the table. Measured on one L40S, block 140,000, two runs per cell:

arm po2 21 po2 22 gain
stock libsecp 446, 446, 446 s (189 seg) 396, 396 s (93 seg) 1.126x
bigint2 48, 49 s (18 seg) 43, 44 s (9 seg) 1.111x
peak VRAM 22,521 / 22,009 MiB 40,661 / 40,345 MiB 1.8x

The gain is consistent across two independent arms, which is what licenses treating ~11% as a property of the setting rather than of one workload. ACCELERATION.md's sweep found ~8% on a 415-segment chunk; this confirms it transfers down to 9 segments — which that file's own caveat said could not be assumed in the other direction.

An obligation the docs did not state

Setting po2 22 means serialising GPU work. At 88% of a 46 GB card, two concurrent proves OOM. hazync run serialises through a lock; a direct host prove-* does not (#97). Reproduced today by running three proves back to back.

Also corrects a doc I wrote yesterday

TOPOLOGY_AND_SETTINGS.md repeated the same wrong default. I introduced that in #196 by consolidating the claim without checking it against the code — the exact failure that PR was about.

And adds the #139 fleet numbers at po2 22

Per-verify ECDSA drops 1,723,407 → 140,044 cycles (12.31x). Chunk work on block 962,000: 14,926 → 2,310 card-seconds.

framing today with #139
(a) aggregate distributes 32 cards 7 cards
(b) segments only, resolution serial 48 cards 10 cards
(c) aggregate serial fails fails at any N
throughput (bounded lag) 29 cards 7 cards

The important part is not the number — it is what it exposes. #139 does not touch the aggregate (no EC verification happens there), so it goes from ~10% of one-card cost to 39%. Under a serial aggregate its 1,575 s alone exceeds the 600 s budget on any hardware, and the entire remaining chunk-side headroom is 7 → 3 cards even with infinitely fast chunks.

⇒ Whether the aggregate distributes is now worth more than every card, po2 and guest-codegen decision combined.

⚠ Assumes hazync#190 has landed — post-#139 ECDSA and Schnorr diverge, and #190's simulation puts an unaware packer's straggler at 2.45x, which would roughly halve the win. Block 962,000 is also only 2.7% Schnorr, which #190 calls "the mildest case available".

⚠ Middle-path @ po2 22 is inferred (scaled by the measured po2 factor), not directly measured; it is labelled as such.

https://claude.ai/code/session_015nyynoX7kXHe7zCF3j6Jbf

SEGMENT_DISTRIBUTION said "22 is the default on CUDA and peaks at ~40.6 GB of
VRAM". Both halves are wrong. main.rs is unambiguous:

    .unwrap_or(if cfg!(feature = "cuda") { 21 } else { 20 })

po2 21 peaks at ~22 GB, measured today at 22,521 and 22,009 MiB on an L40S —
half the 40.6 GB the docs implied, and 40.6 GB is what po2 22 costs, which
you only get by setting the variable.

Two consequences. Any run that did not set HAZYNC_SEG_PO2 was at po2 21, so
figures labelled "at po2 22" elsewhere were only that if someone set it
explicitly. And a fleet's VRAM headroom was being sized against a number
twice the real default.

Measured on one L40S, block 140,000, two runs per cell:

    stock libsecp   446 s (189 seg) -> 396 s (93 seg)   1.126x
    bigint2          48 s ( 18 seg) ->  43 s ( 9 seg)   1.111x

The gain is consistent across two independent arms, which is what licenses
treating ~11% as a property of the setting rather than of one workload.
ACCELERATION's sweep found ~8% on a 415-segment chunk; this confirms it
transfers down to 9 segments, which that file's own caveat said could not be
assumed in the other direction.

Setting po2 22 carries an obligation the docs did not state: serialise GPU
work. At 88% of the card two concurrent proves OOM. `hazync run` serialises
through a lock, a direct `host prove-*` does not (#97) — reproduced today by
running three proves back to back.

Also corrects TOPOLOGY_AND_SETTINGS, which repeated the same wrong default. I
introduced that yesterday by consolidating the claim without checking it
against the code, which is the exact failure the docs audit was about.

And adds the #139 fleet numbers at po2 22: chunk work 14,926 -> 2,310
card-seconds, so 7 cards where it is 32 today. The important part is not the
number but what it exposes — the aggregate is untouched by #139 and becomes
39% of the work, the entire remaining chunk-side headroom is 7 to 3 cards,
and under a serial aggregate ten minutes is unreachable on any hardware.

Claude-Session: https://claude.ai/code/session_015nyynoX7kXHe7zCF3j6Jbf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant