docs: record three evaluation pitfalls that produced wrong conclusions - #533
Open
Ultimate-Storm wants to merge 1 commit into
Open
docs: record three evaluation pitfalls that produced wrong conclusions#533Ultimate-Storm wants to merge 1 commit into
Ultimate-Storm wants to merge 1 commit into
Conversation
Companion to SWARM_FAILURE_MODES.md, which covers how a run fails; this covers how a number misleads. All three actually happened. E1 — macro AUROC ranked centres by how many benign cases their test split happened to contain (0, 1, 2, 2, 12), not by performance. UMCU was called the weakest centre at 0.602 on that basis; excluding the 2-case benign class it is 0.717, and CAM moves 0.710 -> 0.908 on the same correction. RUMC "led" only because it has no benign cases at all, so two classes were averaged instead of three. It is a split artefact: benign is ~10% of UMCU's challenge slice overall. E2 — a checkpoint loaded with the wrong architecture (MST vs 1DivideAndConquer) produced plausible-looking but meaningless output and a false shortcut-learning diagnosis. E3 — accuracy 0.752 alongside malignant AUROC 0.887 hid that the model called malignant 21 times against 37 true cases. The threshold was wrong, not the model. Flags #526 (D2.5 regional fine-tuning) as the next place E1 applies. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to
SWARM_FAILURE_MODES.md— that covers how a run fails, this covers how a number misleads. All three entries actually happened on this project.E1 — macro AUROC does not rank centres at small class support. The per-site table ranked UMCU worst (0.602) and RUMC best (0.857). Both were artefacts: the ordering tracks how many benign cases each test split happened to contain (0, 1, 2, 2, 12), because macro AUROC gives that class a third of the weight regardless of support.
CAM was penalised as hard as UMCU and nobody noticed. RUMC "led" only because two classes were averaged instead of three. It is a split artefact — benign is ~10% of UMCU's challenge slice overall, but only 2 of 20 landed in the 41-case test partition.
Also records that these are the challenge test partition, not any site's training data — 41 of 204 curated exams against a >10,000-exam training contribution. Without that, "UMCU's results are weak" reads as a statement about their data.
E2 — checkpoint loaded with the wrong architecture loads silently and yields meaningless-but-plausible metrics.
E3 — accuracy hid a mis-set decision threshold that under-called malignancy.
Flags #526 (D2.5 regional fine-tuning) as the next place E1 applies, since it compares per-region performance on these same small partitions.
🤖 Generated with Claude Code