Skip to content

scsd: re-emit op 0x02 to the coordinator at the ADD transition (vms-a84d) - #1029

Draft
baron-3dl wants to merge 2 commits into
mainfrom
vms-a84d-cfg2-reemit
Draft

scsd: re-emit op 0x02 to the coordinator at the ADD transition (vms-a84d)#1029
baron-3dl wants to merge 2 commits into
mainfrom
vms-a84d-cfg2-reemit

Conversation

@baron-3dl

Copy link
Copy Markdown
Contributor

The stall

The booted-node cluster join stalls at barrier step 5/12: the second existing member (VAX1) never pushes its disk-DLM rebuild toward the joining OVMX node, so it withholds its own op-0b#5 and the coordinator never releases 0c#5. OVMXJ0 reaches the member flag but is never counted (CLUSTER_NODES stays 2).

Root cause (directly observed on the inter-VAX channel)

The run-1025 pcap was captured on the lab bridge and already contains COORD↔VAX1 traffic. Mining it (three-way grounded against the vax3-2to3 reference):

  • The coordinator emits its op-0x12 "rebuild toward the new member" relay to the other existing member only in response to a FRESH joiner op-0x02 landed AT the transition. Reference: joiner op-0x02 → COORD op-0x12 relay 0.3 ms later → that member rebuilds and clears its barrier step.
  • OVMX's op-0x02 to the coordinator is one-shot on the JOIN_CFG2_DELAY_MS (4.9 s) timer, decoupled from the coordinator's transitions. It did fire COORD's relay once — but for the pre-epoch transition. The OVMX-ADD epoch (0xE) opens afterward and gets no fresh op-0x02, so COORD never relays OVMX into VAX1's rebuild set.

Fix

When the coordinator opens an ADD transition (op-0x09 XITION) while we are the joiner, re-emit a fresh op-0x02 to the coordinator, once per ADD-epoch. Content is byte-identical to the deferred op-0x02 (proven byte-equal to a real joiner's); only the timing changes.

Honesty (INV-6)

No fabricated lock/held state. The gate-clearing member rebuild is a pure cat-0x82 echo OVMX already performs for the coordinator (the member originates its own value blocks; OVMX only reflects them). Two independent reference decodes confirmed the joiner never initiates held-state records.

Scope / safety

Joiner-path-guarded (rx->do_connect && !appeared_after_join), so the member/bystander and OVMX↔OVMX (H2) paths are untouched. No new opcode/enum/TU. All 26 vmsscs unit tests pass (incl. the 141k-frame scsd_wire replay).

⚠ Verification bar

H2 cannot exercise the real COORD→VAX1 relay (the #1026 lesson), so CI-green is necessary-not-sufficient. The decider is the lab re-fire vs vaxlab-2, graded by eye on both consoles via the SDA CSB oracle: COORD relays VAX1 into OVMX's rebuild → VAX1 emits 0b#5 → barrier 5/12→12/12 → CLUB Nodes 2→3 → OVMXJ0 = MEMBER at CLUSTER_NODES=3, cap-denied. Milestone stays un-cut until that is observed sustained on both consoles.

🤖 Generated with Claude Code

baron-3dl and others added 2 commits September 1, 2026 13:53
…84d)

The booted-node cluster join stalls at barrier step 5/12: the second
existing member (VAX1) never pushes its disk-DLM rebuild toward the
joining OVMX node, so it withholds its own op-0b#5 and the coordinator
never releases 0c#5.

Root cause (directly observed on the inter-VAX channel already captured in
the run-1025 pcap; three-way grounded against the vax3-2to3 reference):
the coordinator emits its op-0x12 "rebuild toward the new member" relay to
the other existing member ONLY in response to a FRESH joiner op-0x02
landed AT the transition. In the reference this is exact -- joiner op-0x02
-> COORD op-0x12 relay 0.3ms later -> that member rebuilds and clears its
barrier step. OVMX's op-0x02 to the coordinator is one-shot on the
JOIN_CFG2_DELAY_MS (4.9s) timer (cfg_sent/joiner_cfg2_sent), decoupled
from the coordinator's transitions: it DID fire COORD's relay once, but
for the pre-epoch transition; the OVMX-ADD epoch opens afterward and gets
no fresh op-0x02, so COORD never relays OVMX into VAX1's rebuild set.

Fix: when the coordinator opens an ADD transition (op-0x09 XITION) while
we are the joiner, re-emit a fresh op-0x02 to the coordinator, once per
ADD-epoch. The op-0x02 CONTENT is byte-identical to the deferred one
(proven byte-equal to a real joiner's op-0x02); only its TIMING changes.
No fabricated lock/held state -- INV-6 clean; the gate-clearing member
rebuild is a pure cat-0x82 echo OVMX already performs for the coordinator.

Joiner-path-guarded (rx->do_connect && !appeared_after_join), so the
member/bystander and OVMX<->OVMX (H2) paths are untouched. H2 cannot
exercise the real COORD->VAX1 relay, so CI-green is necessary-not-
sufficient; the lab re-fire vs vaxlab-2 is the actual proof.

All 26 vmsscs unit tests pass (incl. the 141k-frame scsd_wire replay).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhM3QcmFEY3p8YNnHGaYwP
…arrier (vms-a84d)

Lab re-fire #1029b tested the epoch re-emit in isolation (collision landed
on the coordinator's VC, XITION opened, SCSD-I-CFG2REEMIT fired) and
exposed a timing inversion: the re-emit was armed on the XITION but SENT
from the poll-loop choke point, which fires on the JOIN_CFG2_DELAY_MS
(4.9s) cadence -- so it landed at +35.26, ~4s AFTER the barrier had
already run steps 1-5 and stalled at +31.2. The coordinator needs the
fresh op-0x02 BEFORE it fans out the barrier; 4s late it is ignored, no
op-0x12 relay, VAX1 never rebuilds, barrier stuck at 5/12.

The reference (vax3-2to3) sends its relay-triggering op-0x02 0.04s BEFORE
the barrier (+34.7634 op-0x02 -> +34.807 barrier). Match that: send the
fresh op-0x02 SYNCHRONOUSLY in the XITION handler, the instant the ADD
transition opens (~1s before the barrier fan-out), instead of deferring to
the poll loop. Drops the poll-loop block and the cfg2_reemit_pending flag;
keeps cfg2_reemit_epoch as the once-per-epoch guard.

Content still byte-identical to the deferred op-0x02 (INV-6 clean); only
the timing changes. Joiner-path-guarded. All 26 vmsscs unit tests pass.

⚠ Still a candidate: whether a correctly-timed fresh op-0x02 actually
triggers the coordinator's op-0x12 relay is coordinator-internal and
unproven -- the lab re-fire is the arbiter. #1029b could not test it
because the send was 4s late.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhM3QcmFEY3p8YNnHGaYwP
@baron-3dl
baron-3dl marked this pull request as draft September 1, 2026 14:37
@baron-3dl

Copy link
Copy Markdown
Contributor Author

Held as draft — the op-0x02 avenue is refuted. Lab re-fire #1029c-1 confirmed the synchronous timing fix works mechanically (SCSD-I-CFG2REEMIT now fires 0.84s before the barrier, matching the reference), but a correctly-timed fresh op-0x02 to the coordinator produces no op-0x12 relay, no VAX1 rebuild — barrier still stalls at 5/12, CN=2. So op-0x02 does not trigger the transition-time VAX1 relay; the earlier +18.18 correlation was coincidental. Both parts of this PR (timing + would-be coordinator routing) are moot against a refuted mechanism. Holding as draft pending identification of the real coordinator-internal relay trigger; this branch will be reworked or closed once that lands. Cap-denial passed on every run — anti-fabrication guarantee intact throughout.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant