Skip to content

test: a committed mutant list, run nightly - #52

Merged
cubehouse merged 1 commit into
mainfrom
ci/nightly-mutation
Sep 28, 2026
Merged

cubehouse merged 1 commit into
mainfrom
ci/nightly-mutation

Conversation

@cubehouse

Copy link
Copy Markdown
Member

The same job the Python SDK now carries (ThemeParks_Python#30), and for the same reason.

An author-written mutant list contains the mutations the author's tests already catch. One scored 18/18 on this package while an independent sweep found ten survivors. That is grading my own homework, and the score was the reassuring kind.

So the list is committed and reviewable — test/mutation/mutants.json, twenty entries, each carrying why it matters. Most name a defect that reached a real file:

  • the write error finish() discarded, so ENOSPC printed done: N rows and exited 0 with a truncated file
  • the stream with no 'error' listener, which killed a six-park run on the first park
  • the checkpoint taken from the newest row rather than the page boundary
  • the bare \r that split one CSV row into two
  • the CJK query that listed all 127 parks because normalize('東京') is ''

Nightly and non-gating: a survivor is information, not a reason to block a merge. A mutant whose find no longer matches counts as a failure too — a stale mutant has been silently testing nothing, which is the same failure mode as a test that cannot fail.

One mutant targets the generated column list, so the shared csv_contract.json fixture is exercised as a mutation as well as a test: renaming a column there must turn the suite red, or the two SDKs can drift apart again.

20/20 killed.

🤖 Generated with Claude Code

The same job the Python SDK now carries, and for the same reason: an
author-written mutant list contains the mutations the author's tests already
catch. One scored 18/18 on this package while an independent sweep found ten
survivors. That is grading my own homework, and the score was the reassuring
kind.

So the list is committed and reviewable -- what is being checked, and more
usefully what is not. Each entry carries why it matters, most of them naming a
defect that reached a real file: the write error `finish()` discarded, the stream
with no 'error' listener, the checkpoint taken from the newest row, the bare `\r`
that split a CSV row in two, the CJK query that listed all 127 parks.

Nightly and non-gating: a survivor is information, not a reason to block a merge.
A mutant whose `find` no longer matches counts as a failure too, because a stale
mutant has been silently testing nothing -- the same failure mode as a test that
cannot fail.

One of the mutants is on the GENERATED column list, so the shared
csv_contract.json fixture is exercised as a mutation as well as a test: renaming
a column there must turn the suite red, or the two SDKs can drift apart again.

20/20 killed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cubehouse
cubehouse merged commit fa4db78 into main Sep 28, 2026
4 checks passed
@cubehouse
cubehouse deleted the ci/nightly-mutation branch September 28, 2026 18:21
@cubehouse cubehouse mentioned this pull request Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant