Flaky e2e: workflows intermittently stuck in "untriggered" on the HA stack

[gustavobertoi]

Summary

On the multi-node e2e stack (docker compose --profile e2e, 3 nodes + Postgres), a freshly
triggered workflow intermittently stays in status untriggered until the test times out. The
trigger HTTP call is accepted (a workflowId is returned) but the workflow actor is apparently
not spawned/claimed in time, so it never advances.

Evidence it is a pre-existing flake (not a specific PR)

• The same commit df827dc (feat(secrets): per-execution environment scoping for secret resolution (ADR-0031 Phase 3, PR A) #83) both passed and failed the E2E workflow within ~2h
  (passed 19:46, failed 21:40/21:44) — non-deterministic.
• The identical still in status "untriggered" failure occurred on 16867aa (before the
  ADR-0031 Phase 3 work), affecting multiple unrelated workflows
  (TestWorkflowResilienceSuite/TestTimeout_FollowsRecoveryPath, retry, etc.), not just one.
• Timers are not the cause: TestTimedCondition_CompletesAfterTimer (also a 3s timer) passes.

Example failure: TestWorkflowExecutionSuite/TestSumRandBranch_Finishes —
workflow ... still in status "untriggered" after 1m0s
(run https://github.com/open-source-cloud/fuse/actions/runs/26914935708).

Likely area

The HA trigger → spawn/claim path in the clustered runtime (workflow supervisor spawn, claim
repository / durable execution, ADR-0010), where a triggered workflow occasionally is not picked
up by a node. Independent of the secrets/settings work.

Stopgap (separate PR)

The e2e suite now re-triggers a workflow that is stuck specifically in untriggered
(TriggerAndWaitTerminal), so the flake no longer reds the build while the root cause is
investigated. Terminal states (including error) are never retried, so real failures still surface.

Root-cause TODO

• Reproduce against the 3-node --profile e2e stack and capture per-node logs for a stuck
  workflow (was the TriggerWorkflow message received? was a WorkflowInstanceSupervisor spawned?
  did claim acquisition stall?).
• Decide whether trigger delivery / spawn needs an ack/retry or claim-timing fix.