Skip to content

fix(resolution): #870 tone is not a property — benefit-vs-limitation contradiction FPs measured to zero - #879

Merged
jasonssdev merged 1 commit into
mainfrom
fix/870-benefit-limitation-arm
Aug 25, 2026
Merged

fix(resolution): #870 tone is not a property — benefit-vs-limitation contradiction FPs measured to zero#879
jasonssdev merged 1 commit into
mainfrom
fix/870-benefit-limitation-arm

Conversation

@jasonssdev

Copy link
Copy Markdown
Owner

Closes #870

What this ships

One sentence inserted into the contradiction judge's rubric (resolution/contradiction.py), after #558's antonymy carve-out: benefit-vs-limitation names DIFFERENT properties, two bodies acknowledging the same limitation AGREE about it, and contradicts needs incompatible values for one property — never opposite tone. Nothing else in the prompt moves.

Plus the measurement that justified it: 6 new fixture pairs in evals/contradictions/ (benefit-limitation ×4, the first mirroring the wild pair exactly including its internal "Sin embargo…" integration; evaluative-contradiction ×2 as the wash-out guard), new headline metrics in the runner, and both 15-run stored arms.

Measured (qwen3:8b, 15 runs/arm, production client settings, 18-pair fixture)

arm benefit-limitation FP antonym FP TP retention evaluative guard accuracy stability
baseline 0.15 0.32 1.00 1.00 0.86 0.95
treatment (ADOPTED) 0.00 0.00 1.00 1.00 1.00 1.00

The baseline's class failure sits entirely in the wild-mirror pair: wrong 14 of 15 runs (9 contradicts at 0.95, 5 uncertain). The failing shape is precise — both bodies acknowledge the SAME limitation, one in passing and one in depth, so the claims agree and only the tone opposes.

Bonus, unlooked-for: antonym FP fell 0.32 → 0.00, closing the allowlist/denylist residual #558 recorded as unfixable at this model size. The treatment arm is 270/270 with every pair at stability 1.00.

TDD

test_the_rubric_separates_tone_from_property written RED first, GREEN after the insert, and mutation-verified (sentence removed → test fails; byte-exact restore + __pycache__ purge → green). Full suite 5653 passed / 1 skipped.

Disclosures

@jasonssdev
jasonssdev merged commit 907eda3 into main Aug 25, 2026
7 checks passed
@jasonssdev
jasonssdev deleted the fix/870-benefit-limitation-arm branch August 25, 2026 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

contradictions: a benefit and a limitation of one technique judged CONTRADICTS at 0.95

1 participant