Skip to content

Stop the stale rescheduler cloning live and undeliverable matches (#174) - #175

Open
razam-sherwani wants to merge 1 commit into
stagingfrom
fix/174-stale-rescheduler
Open

Stop the stale rescheduler cloning live and undeliverable matches (#174)#175
razam-sherwani wants to merge 1 commit into
stagingfrom
fix/174-stale-rescheduler

Conversation

@razam-sherwani

Copy link
Copy Markdown
Contributor

Closes #174.

Problem

StaleGameMatchRescheduler (@Scheduled(fixedRate=60000)) requeued everything findStaleWaitingMatches returned — and that query selected status IN ('waiting','in_progress')without cancelling the original. A slow or wedged match got re-enqueued every minute past the 120-min threshold, times_scheduled climbed, and the rescheduler multiplied load under the exact stress it fires in. This was the automatic-load half of last semester's queue, and it recurs for any future tournament.

Fix

  • Sweep only waiting. A match becomes in_progress only once a worker emits a "started" update (GameMatchResultHandler:46), i.e. a worker owns it — cloning it is the core defect. A genuinely wedged in_progress match is the per-match watchdog's job (ByteFight-GT/cs3600_2026#35), which this issue is explicitly paired with; that division is intentional, not a gap.
  • Bound retries. Once a match has been scheduled gamematch.max-reschedules times (default 5) and is still waiting, mark it failed instead of requeuing, via the existing finalizeMatchResult + failed status. An undeliverable match can no longer loop forever.
  • Wire the dead flag. GameMatchProperties.requeueStale was defined and set true in application.yml but read nowhere. The sweep now honours it — a real kill-switch.

No schema change (times_scheduled and MatchStatus.failed already exist).

Tests

New StaleGameMatchReschedulerIT: in_progress-past-threshold is not requeued; waiting-past-threshold is requeued once and times_scheduled bumps; at max-reschedules it flips to failed; requeue-stale: false short-circuits the sweep. GameMatchServiceIT still green.

Verified locally (no CI here): ./gradlew test --tests StaleGameMatchReschedulerIT --tests GameMatchServiceIT on a JDK-17 toolchain, Testcontainers Postgres+RabbitMQ. Independent of #172 and #173.

🤖 Generated with Claude Code

findStaleWaitingMatches selected status IN ('waiting','in_progress') and the
rescheduler requeued each without cancelling the original, so a wedged match
was re-enqueued every minute past the threshold and times_scheduled climbed —
load multiplied under the exact stress the sweep fires in.

- Sweep only 'waiting'. A match becomes 'in_progress' only once a worker emits
  a "started" update (GameMatchResultHandler), so requeuing one clones live
  work; a wedged in_progress match is the per-match watchdog's job
  (ByteFight-GT/cs3600_2026#35), not this sweep's.
- Fail a match instead of requeuing it once it has been scheduled
  gamematch.max-reschedules times (default 5), via the existing
  finalizeMatchResult + failed status, so an undeliverable waiting match can't
  loop forever.
- Honour the existing gamematch.requeue-stale flag, which was defined and set
  true but read nowhere — it's now a real kill-switch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stale rescheduler re-publishes in-flight matches, multiplying load

1 participant