Skip to content

Variant-node sharing is now unconditional (nodes -34.6%, accuracy neutral) - #94

Merged
adamjohnwright merged 6 commits into
mainfrom
feat/share-variant-nodes-default
Sep 21, 2026
Merged

adamjohnwright merged 6 commits into
mainfrom
feat/share-variant-nodes-default

Conversation

@adamjohnwright

Copy link
Copy Markdown
Contributor

Phase 1 keyed entity uuids on the virtual reaction, so a participant common to every variant of a Reactome reaction was photocopied once per set-member combination — HDR's BCDX2 complex existed as 33 identical nodes. Keying on the Reactome reaction is what the data says. The flag existed only to measure that the correction costs nothing.

It does not. Pre-registered in deltasignal specs/020-variant-node-sharing before the regeneration; all four predictions hold.

Size (P1)

before after
nodes 108,229 70,738 −34.6%
edges 285,497 224,189 −21.5%
cyclic components 210 204
nodes in cycles 8,407 8,131
largest component 675 675 unchanged

Honest caveats, all recorded in the spec: sharing is a literal no-op in 47 of 92 pathways, the median reduction among the 45 that move is only 6.7%, and the catalog figure is carried by a few (Fanconi Anaemia −83.5%, DNA Repair −61.8%). About a third of catalog nodes are dissociation sinks with no out-edges, so the −21.5% edge figure is the more conservative read.

The new-cycle risk was checked, not assumed. Merging two variant strands could in principle weld a cycle neither had, which would undo the largest accuracy win this project has (specs/018 unwelding, held-out +173). Measured across all 92 pathways: 0 gained cyclic nodes; 4 lost a component.

Accuracy (P2–P4)

29 of 24,100 predictions change — 0.12%, in 3 pathways (DNA Repair 14, RUNX2 10, MET 5). Only DNA Repair has a nonzero net; the other two fixed exactly as many as they broke.

before after
curator 20414/24100 = 84.71%, mF1 0.8137 20400/24100 = 84.65%, mF1 0.8125
experimental 611/849 = 71.97%, mF1 0.6451 609/849 = 71.73%, mF1 0.6432
false change 1,150 1,150

Under perturbation-set conditioning, held-out net is 0 with zero discordant cases on both axes. Sharing changes which nodes a gene resolves to, so 4,094 of 24,100 cases are not like-for-like and are dropped; among the rest nothing moves at all.

The net −14 is in a tuning pathway and is not quoted as an effect (P5).

Verification that the default reproduces what was measured

Regenerated two pathways with no flag set and compared against the flagged catalog:

pathway flagged run new default no-sharing control
HDR 799 nodes / 3,123 edges 799 / 3,123 1,719 / 3,648
DNA Repair 4,756 / 14,487 4,756 / 14,487 12,442 / 24,332

The control catalog and this one were built from byte-identical generator trees (13847fa is the merge commit whose second parent is 79feca7; git diff between them is empty), so the generator is not a confound.

Why no flag

Per the flag-expiry policy: a flag whose question is answered is a second code path nobody measures. Setting LNG_SHARE_VARIANT_NODES now raises rather than being ignored, so a stale value in a benchmark script cannot silently measure something else.

Also in this PR

LNG_BOUNDARY_EXPANSION, LNG_COMPOSITION_EDGES and LNG_EMIT_ONE_SIDED are added to _FINGERPRINTED_ENV. All three change emitted content, and that list's own comment already promises that flipping a fingerprinted flag invalidates stale caches "instead of silently reusing a network built under the old setting" — which is exactly what these three could do.

Docstring fix: the BCDX2 example spans 6 reactions, not 5.

Not fixed here

Adversarial review found that ~60% of nodes with in-edges are exact structural twins (identical in-edge and out-edge sets). Sharing removes 38% of them but leaves the rate untouched, because it does not merge reaction nodes. Filed in specs/020 as a separate finding with an explicit warning: the two nearest interventions already measured are negative and load-bearing (specs/012, −15 and −61 held-out), so this needs its own pre-registration, not a patch.

tests/test_variant_node_sharing.py  11 passed
full suite                          1022 passed, 1 pre-existing MHC failure
ruff check src/ bin/                clean
mypy src/                           clean

🤖 Generated with Claude Code

adamjohnwright and others added 6 commits September 21, 2026 15:06
…rror

The flag existed to ask one question -- does refusing to reuse a node the root
complex can reach lose anything? -- and that question is answered: held-out
+173 conditioned, p<1e-4, false change 1437->1150 (deltasignal specs/018). A
flag whose question is settled is a second code path nobody measures.

Removing it outright would make a stale LNG_BOUNDARY_LEAF_REUSE=any in an
environment or a benchmark script silently measure the default while the
operator believed they were measuring the old mode -- exactly the silent-input
bug class this project has hit four times. So the name is kept as a rejection:
setting it to anything raises, naming the spec. The two mode tests are replaced
by one that asserts that error for three values, including the two that used to
be valid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t gap

Phase 1 keyed entity uuids on the VIRTUAL reaction, so a participant common to
every variant of a Reactome reaction was photocopied once per set-member
combination. Keying on the Reactome reaction instead is what the data says;
the flag existed only to measure that the correction costs nothing.

Measured on the 92-pathway catalog (deltasignal specs/020):

  nodes             108,229 -> 70,738   (-34.6%)
  edges             285,497 -> 224,189  (-21.5%)
  pathways gaining cyclic nodes            0   (does not undo specs/018)
  predictions changed        29 of 24,100 (0.12%, 3 pathways)
  held-out net, both axes                  0   (zero discordant cases)
  false change             1,150 -> 1,150

Verified that the new default reproduces the measured catalog exactly: HDR
799 nodes / 3,123 edges and DNA Repair 4,756 / 14,487, identical to the
flagged run.

Setting LNG_SHARE_VARIANT_NODES now raises rather than being ignored, so a
stale value in a benchmark script cannot silently measure something else.

Also adds LNG_BOUNDARY_EXPANSION, LNG_COMPOSITION_EDGES and LNG_EMIT_ONE_SIDED
to _FINGERPRINTED_ENV. All three change emitted content, and the list comment
already promised that flipping any fingerprinted flag invalidates stale caches
"silently reusing a network built under the old setting" -- which is exactly
what these three could do.

Docstring fix: the BCDX2 example is 6 reactions, not 5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adversarial review of the diff found the guard was decorative on both halves.

It had NO runtime coverage. Its only test asserted that a string appeared in
inspect.getsource(), which passes whether or not the guard runs -- mutating it
to `if False and ...` left the whole suite green, and exporting the flag and
running the suite gave the baseline result, because no test reaches that line.
Replaced with a shared _reject_removed_env() helper, unit-tested with
monkeypatch and pytest.raises across five values including "0" and "".

And it could not stop a run even when it fired. bin/create-pathways.py catches
every per-pathway exception, counts it, and returns None -- so a stale flag
meant every pathway raised, nothing was written, the PREVIOUS catalog stayed on
disk, and the shell saw exit 0. A benchmark reading that directory would have
silently scored the old catalog. Now checked once at the top of main() before
any work, and the command exits 1 if any pathway failed. Six other error paths
in main() also returned 0 on failure; all now exit 1.

All three mutations are caught by the new tests: dead guard body, missing
entry-point call, and restored exit-0-on-failure.

The error message also claimed a guarantee the code does not provide. _succ is
a snapshot taken before the emission loop and is not updated as this function
appends its own assembly edges, so a later root complex can still be handed a
leaf that an earlier one made reachable. The reviewer constructed the weld.
That bug is PRE-EXISTING and identical on both sides of this flag removal, so
the comment and message now state the rule precisely rather than overclaiming;
fixing it changes emitted networks and would invalidate the measurement this
rule rests on, so it is tracked separately.

Also: a conftest clears removed names for the whole suite (this diff otherwise
turned a stale export into 10 spurious failures, up from 1), unused monkeypatch
fixtures dropped, and PIPELINE_SETTINGS.md gains a removed-flags table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n fail

Guard now routes through the shared, tested _reject_removed_env() from #93,
so a stale LNG_SHARE_VARIANT_NODES is rejected at process entry before any
database work and the command exits non-zero. The previous guard had no
runtime coverage at all: its only test asserted a string appeared in
inspect.getsource(), which passed with the guard mutated to `if False and ...`
and broke the moment the guard was refactored. Deleted; tests/test_removed_
env_flags.py now parametrizes over both removed names.

Three factual corrections to the docstring:

- The summary line said "one key per (entity, REACTOME reaction, role)". That
  is FALSE and a reader acting on it would break Phase 2, which looks entities
  up by VIRTUAL reaction. The key is unchanged; the shared thing is the VALUE.
- "-83 and -117 held-out" was wrong on both numbers. specs/016 P12 gives min
  over copies -85 held-out (p 0.0021) and max over copies -322 (p<0.0001);
  -117 is a DSB-Repair-only ACCURACY delta, not a held-out figure.
- The size result now carries the caveat the spec insists on: the -34.6% node
  figure flatters it (no-op in 47 of 92 pathways, median mover 6.7%), the
  -21.5% edge figure is the conservative read, and the zero held-out net comes
  from conditioning that drops 4,094 non-comparable cases while both axes fall
  slightly unconditioned.

The two boundary tests asserted a mechanism that cannot occur -- the variant
cache could only ever store the boundary uuid, so deleting the skip left every
uuid identical. They now assert on the shared/distinct stats, which is what
actually distinguishes the implementations, and the mutation is caught.

Re-verified after all edits: 1500 randomized inputs give 0 differences against
main's share_variants=True path, and HDR (799/3,123) and DNA Repair
(4,756/14,487) reproduce the measured catalog exactly.

Fingerprint honesty: only LNG_EMIT_ONE_SIDED is a genuine cache gap; the other
two are read inside a function that re-runs every time, so they buy provenance
at the cost of a re-fetch. Kept deliberately, now documented as such.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…des-default

# Conflicts:
#	PIPELINE_SETTINGS.md
#	src/logic_network_generator.py
@adamjohnwright
adamjohnwright merged commit f2842bc into main Sep 21, 2026
4 checks passed
@adamjohnwright
adamjohnwright deleted the feat/share-variant-nodes-default branch September 21, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant