Stop the stale rescheduler cloning live and undeliverable matches (#174) - #175
Open
razam-sherwani wants to merge 1 commit into
Open
Stop the stale rescheduler cloning live and undeliverable matches (#174)#175razam-sherwani wants to merge 1 commit into
razam-sherwani wants to merge 1 commit into
Conversation
findStaleWaitingMatches selected status IN ('waiting','in_progress') and the
rescheduler requeued each without cancelling the original, so a wedged match
was re-enqueued every minute past the threshold and times_scheduled climbed —
load multiplied under the exact stress the sweep fires in.
- Sweep only 'waiting'. A match becomes 'in_progress' only once a worker emits
a "started" update (GameMatchResultHandler), so requeuing one clones live
work; a wedged in_progress match is the per-match watchdog's job
(ByteFight-GT/cs3600_2026#35), not this sweep's.
- Fail a match instead of requeuing it once it has been scheduled
gamematch.max-reschedules times (default 5), via the existing
finalizeMatchResult + failed status, so an undeliverable waiting match can't
loop forever.
- Honour the existing gamematch.requeue-stale flag, which was defined and set
true but read nowhere — it's now a real kill-switch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #174.
Problem
StaleGameMatchRescheduler(@Scheduled(fixedRate=60000)) requeued everythingfindStaleWaitingMatchesreturned — and that query selectedstatus IN ('waiting','in_progress')— without cancelling the original. A slow or wedged match got re-enqueued every minute past the 120-min threshold,times_scheduledclimbed, and the rescheduler multiplied load under the exact stress it fires in. This was the automatic-load half of last semester's queue, and it recurs for any future tournament.Fix
waiting. A match becomesin_progressonly once a worker emits a "started" update (GameMatchResultHandler:46), i.e. a worker owns it — cloning it is the core defect. A genuinely wedgedin_progressmatch is the per-match watchdog's job (ByteFight-GT/cs3600_2026#35), which this issue is explicitly paired with; that division is intentional, not a gap.gamematch.max-reschedulestimes (default 5) and is stillwaiting, mark itfailedinstead of requeuing, via the existingfinalizeMatchResult+failedstatus. An undeliverable match can no longer loop forever.GameMatchProperties.requeueStalewas defined and settrueinapplication.ymlbut read nowhere. The sweep now honours it — a real kill-switch.No schema change (
times_scheduledandMatchStatus.failedalready exist).Tests
New
StaleGameMatchReschedulerIT: in_progress-past-threshold is not requeued; waiting-past-threshold is requeued once andtimes_scheduledbumps; atmax-reschedulesit flips tofailed;requeue-stale: falseshort-circuits the sweep.GameMatchServiceITstill green.Verified locally (no CI here):
./gradlew test --tests StaleGameMatchReschedulerIT --tests GameMatchServiceITon a JDK-17 toolchain, Testcontainers Postgres+RabbitMQ. Independent of #172 and #173.🤖 Generated with Claude Code