Give the challenge sweeps an explicit timeout - #20
Merged
Merged
Conversation
A Kotest `TestConfig(timeout = ..)` alone would not have fixed this. The failure on #18 was `kotlinx.coroutines.test.UncompletedCoroutinesError`, raised by runTest's own 60s default, which a Kotest timeout cannot raise. `testApplication` is `runTestWithRealTime { runTestApplication(..) }`, and that wrapper applies the runTest default. `runTestApplication` is the same public entry point without it, and since a Kotest test body is already a coroutine it can be awaited directly. That removes the hidden 60s ceiling and leaves the declared 5-minute timeout as the only governing limit. Verified the timeout actually has teeth rather than assuming it: setting it to 1s fails both sweeps with kotlinx.coroutines.TimeoutCancellationException: Coroutine "spec-scope-.." timed out waiting for 1000 ms which is Kotest's spec-scope timeout, not UncompletedCoroutinesError — confirming both that the value is honored and that the mechanism changed. Restored to 5 minutes, where all 5 tests pass and `make lint` is clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
pambrose
added a commit
that referenced
this pull request
Sep 22, 2026
Bundles the two test-reliability fixes that landed after 1.0.1 (#19, #20) with a dependency refresh and the documentation needed to cut the release. Dependencies: readingbat-core 3.3.1 -> 3.4.0, Ktor 3.5.1 -> 3.6.0, Kotest 6.2.3 -> 6.2.5, kotlinter 5.6.0 -> 5.7.0, detekt 2.0.0-alpha.5 -> 2.0.0-alpha.6, and the versions plugin 0.57.0 -> 0.64.0. The Gradle wrapper moves 9.6.1 -> 9.7.1. Kotlin stays at 2.4.10, common-utils at 3.2.2, and the JVM toolchain at 25. Adds the repository's first .gitattributes. gradlew.bat had been stored with LF endings, which cmd.exe mis-parses around labels and multi-line set blocks -- a latent break that Linux-only CI would never have surfaced. Stored blobs are now LF, *.bat is checked out CRLF, and *.jar is pinned binary. `git add --renormalize .` was a no-op, so no existing blob changed. Documentation was audited against the repo rather than just appended to: - CHANGELOG.md: [Unreleased] becomes [1.1.0], matching how 1.0.1 was cut, plus the previously undocumented dependency and wrapper bumps. - RELEASE_NOTES.md: v1.1.0 narrative covering the runTest 60s-ceiling diagnosis. - README.md: the Testing section was stale -- it still described the suite as using testApplication, which is exactly what #20 moved away from. - CLAUDE.md, llms.txt: record the two constraints that are easy to break when editing the suite (await runTestApplication directly; a new language needs its own sweep). Verified with `./gradlew --rerun-tasks check` on the new dependency set: all 5 tests pass and both lint gates are clean. `./gradlew properties` reports 1.1.0. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #19, which split the challenge sweep per language but left the worst body at roughly 34s of a 60s budget by CI extrapolation.
The obvious fix would not have worked
Adding
TestConfig(timeout = ..)and stopping there would have looked right while changing nothing about the actual failure mode. The #18 failure was:That comes from
runTest's own default timeout, which is a separate mechanism from Kotest's test-case timeout. A Kotest timeout cannot raise it.Reading the Ktor 3.5.1 source:
testApplicationtakes no timeout parameter — therunTestWithRealTimewrapper is what imposes the 60s default.runTestApplicationis the same public entry point without the wrapper, and because a Kotest test body is already a coroutine it can be awaited directly.Change
The two sweeps await
runTestApplicationand declareTestConfig(timeout = CHALLENGE_SWEEP_TIMEOUT)(5 minutes). The hidden 60s ceiling is gone, so the declared timeout is the only governing limit.Verified the timeout has teeth
Rather than assume, I temporarily set it to
1.seconds:Test all Java challengesTest all Kotlin challengesThat is Kotest's spec-scope timeout, not
UncompletedCoroutinesError— confirming both that the declared value is honored and that the underlying mechanism actually changed. Restored to 5 minutes afterward.Verification at 5 minutes
./gradlew test --rerun-tasks)make lintclean (kotlinter + detekt)Note
runTestApplicationcarries a Ktor source comment reading "not really needed outside ktor probably". It is public API, but that phrasing suggests JetBrains sees it as niche, so it is worth knowing this is the coupling point if a future Ktor upgrade changes the test entry points.Test with correct answersandTest individual challengesstill use plaintestApplication; they complete in under 0.1s and need no headroom.🤖 Generated with Claude Code