Skip to content

fix(workflows): make execution claims race-safe - #468

Draft
vladbisceanu wants to merge 7 commits into
useplunk:nextfrom
vladbisceanu:upstream/139-race-safe-execution
Draft

fix(workflows): make execution claims race-safe#468
vladbisceanu wants to merge 7 commits into
useplunk:nextfrom
vladbisceanu:upstream/139-race-safe-execution

Conversation

@vladbisceanu

@vladbisceanu vladbisceanu commented Aug 27, 2026

Copy link
Copy Markdown

Problem

Workflow execution relied on read-then-write state changes. Duplicate workers could both perform one step side effects, and an event and its timeout could both observe the same WAITING row before either outcome persisted ownership. Re-entry checks also ignored WAITING executions.

What changed

  • Add a unique constraint for one step row per workflow execution and step.
  • Add a partial unique index allowing only one RUNNING or WAITING execution per workflow and contact pair.
  • Claim the workflow row and step row in one transaction using compare-and-swap predicates.
  • Make event and timeout continuations compete for the same durable wait claim.
  • Reject stale delayed jobs and stale next-step writers.
  • Treat WAITING executions as active in API and event-driven enrollment.
  • Preserve repair of previously claimed event and timeout continuations.

Both indexes are created concurrently. The migrations deliberately fail on pre-existing invariant violations instead of deleting or rewriting workflow history.

Stack

This PR builds on the open workflow durability stack targeting next:

The P7-specific commit is 83fb471. Its patch ID matches the reviewed fork commit exactly. Dependency changes will disappear as those PRs merge and this branch is rebased.

Verification

  • API and dependency builds passed on Node 24.
  • Focused service and durability tests: 164 passed.
  • Race-only adversarial run: 7 passed.
  • Full API suite on this next stack: 36 files, 945 tests passed.
  • Fresh PostgreSQL schema applied all migrations.
  • A P6-to-P7 upgrade probe preserved a seeded WAITING workflow and step and created both valid indexes.
  • Final stable-head review and diff check passed.

Tracks setup-works#139.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant