test: a committed mutant list, run nightly - #52
Merged
Merged
Conversation
The same job the Python SDK now carries, and for the same reason: an author-written mutant list contains the mutations the author's tests already catch. One scored 18/18 on this package while an independent sweep found ten survivors. That is grading my own homework, and the score was the reassuring kind. So the list is committed and reviewable -- what is being checked, and more usefully what is not. Each entry carries why it matters, most of them naming a defect that reached a real file: the write error `finish()` discarded, the stream with no 'error' listener, the checkpoint taken from the newest row, the bare `\r` that split a CSV row in two, the CJK query that listed all 127 parks. Nightly and non-gating: a survivor is information, not a reason to block a merge. A mutant whose `find` no longer matches counts as a failure too, because a stale mutant has been silently testing nothing -- the same failure mode as a test that cannot fail. One of the mutants is on the GENERATED column list, so the shared csv_contract.json fixture is exercised as a mutation as well as a test: renaming a column there must turn the suite red, or the two SDKs can drift apart again. 20/20 killed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The same job the Python SDK now carries (ThemeParks_Python#30), and for the same reason.
An author-written mutant list contains the mutations the author's tests already catch. One scored 18/18 on this package while an independent sweep found ten survivors. That is grading my own homework, and the score was the reassuring kind.
So the list is committed and reviewable —
test/mutation/mutants.json, twenty entries, each carrying why it matters. Most name a defect that reached a real file:finish()discarded, so ENOSPC printeddone: N rowsand exited 0 with a truncated file'error'listener, which killed a six-park run on the first park\rthat split one CSV row into twonormalize('東京')is''Nightly and non-gating: a survivor is information, not a reason to block a merge. A mutant whose
findno longer matches counts as a failure too — a stale mutant has been silently testing nothing, which is the same failure mode as a test that cannot fail.One mutant targets the generated column list, so the shared
csv_contract.jsonfixture is exercised as a mutation as well as a test: renaming a column there must turn the suite red, or the two SDKs can drift apart again.20/20 killed.
🤖 Generated with Claude Code