bench(pk): add same-window interleaved A/B harness mode (--pk-ab) - #387
Merged
Conversation
Runs two default-config variants as alternating rep pairs (A1,B1,A2,B2,...) and reports the per-rep paired median ratio B/A per phase, so machine drift hits both arms of a pair and cancels out — the reliable way to attribute default-vs-tuned deltas. Variant config builder factored out of --pk-default (BuildPkDefaultVariantConfig) and passed per arm; arms selectable via SHARPCOREDB_PK_AB_ARM_A/_B, reps via SHARPCOREDB_BENCH_REPS. Smoke (1 rep, pure default vs 'plain'/NoEncryptMode=true, same window): UPDATE 1.41x, DELETE 1.28x, INSERT 1.21x, READ 1.10x (B over A). Documented in docs/benchmarks/default-config-pk.md.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Samenvatting
P3c: same-window interleaved A/B-harness (
--pk-ab).Runs twee default-config-varianten als alternerende rep-paren (A1,B1,A2,B2,...) en rapporteert per
fase de per-rep paired median ratio B/A — machine-drift raakt beide armen van een paar en
valt weg. Dit maakt default-vs-tuned deltas betrouwbaar toewijsbaar (de beloofde volgende stap uit
#386). Variant-config-builder is uit
--pk-defaultgefactord (BuildPkDefaultVariantConfig) enwordt per arm meegegeven; armen via
SHARPCOREDB_PK_AB_ARM_A/_B, reps viaSHARPCOREDB_BENCH_REPS.Smoke (1 rep, pure default vs
plain/NoEncryptMode=true, zelfde venster): UPDATE 1.41x,DELETE 1.28x, INSERT 1.21x, READ 1.10x (B over A). Richting consistent met eerdere medians,
nu in één venster gemeten. Gedocumenteerd in
docs/benchmarks/default-config-pk.md.Validatie