Declare a metric's scale only where something backs the claim

[borgr]

What / source

A metric's declared bounds are a claim about the scale its score was computed on. Four ways that claim was being made without anything to back it, and one check so the first of them cannot recur.

fields.md already states every rule here. Nothing enforced any of them, and the validator never consults the registry at all.

1. A declared scale contradicting the entry it cites. lm-eval published sacrebleu's BLEU as 31.4 under the canonical id bleu, declaring max_score: 100.0, while that entry says 1.0. The cross-source join metric_id exists for would average it against another source's 0.314, with metric_unit the only thing separating them. bounds_conflicts in tools/verify_metric_ids.py now compares every scale the three converter bounds tables declare against its entry, and the verifier exits non-zero on a disagreement. An entry with no max_score asserts nothing and constrains nothing — 117 of the registry's 397 entries are in that state.

2. An observed maximum declared as a ceiling, at 22 sites across five adapters and one converter. Two of them had a function called compute_observed_max_scores. The ceiling moved whenever a new model entered the leaderboard, so the same metric_id declared a different scale on every refresh and two records of one metric stopped being comparable — and three of the five adapters run on the daily cron, so it happened without anyone choosing it. Costs, durations, latencies, throughputs, a token count, an Elo rating, and a rank whose maximum was only that day's entrant count now declare no ceiling.

3. BLEU converted onto the scale its entry declares. BLEU is a brevity penalty times a geometric mean of clipped precisions, so the quantity is on [0, 1] and sacrebleu's ×100 is a rendering of it. Percent to proportion is provable from the definition, so the record declares [0, 1], carries canonical_rescale_factor and the source figure, and the standard error converts with the score — a spread is in the score's units, which is the shape of the bug Put a rescaled score's reported spread on the score's scale #276 fixed in the Papers with Code adapter.

4. make_metric_config no longer guesses. It defaulted min_score=0.0, max_score=1.0 and built MetricConfig before assigning either, and the model rejects continuous without min_score — so its documented default path raised and it could not emit a continuous metric at all. Both bounds are now the caller's, with a message naming what to pass including float('inf').

Review lane

• Fast
• Needs a human — a material change in outcome. The same input now produces different records for BLEU and for every metric that had an observed-maximum ceiling.

Design agreed in: nothing prior, and I want to be straight about that. Every change here applies a rule fields.md already states rather than proposing a new one, and each is a correction of a clear violation — but the outcome change is real and wants a maintainer's eye. Context for how these were found is on #250.

Checklist

• uv run pytest tests green — 1125 passed, 1 skipped
• uv run ruff check clean
• uv run --extra all python tools/verify_metric_ids.py --seed <registry>/seed/metrics.yaml reports 0 metrics whose declared scale contradicts their entry, with the registry unchanged
• offline tests added: 7 for the verifier, which had none, plus 5 for the schema helper and 4 for the new bounds behaviour
• count of already-published records this would change: not measured. This is the rung-1 question and I have not answered it. Every BLEU record from the lm-eval converter and every record carrying an observed-maximum ceiling would differ on a re-run. Happy to measure it before this merges if you want the number first.

Decisions & coverage

• Decision / where: Artificial Analysis' three composite indices — intelligence, coding, math.
  Chose no bounds and no score_type, with bounds_status: unknown / instead of: declaring infinity.
  Confidence: medium. The source publishes no range and the definition fixes none, and I could not establish one offline. fields.md says "not provided" beats a guess, and infinity is its own claim. If AA documents these as 0–100 it is a one-line change back.
  General? yes — the distinction between unbounded and range unknown applies to any leaderboard metric.

• Decision / where: the registry's bleu entry.
  Chose leave it at [0, 1] and convert the value / instead of: widening the entry to [0, 100].
  Confidence: high. [0, 1] is the quantity's actual range; 0–100 is a percentage rendering. I drafted the entry change first and reverted it.
  General? no.

• Decision / where: chrf and ter.
  Chose leave them on the harness's percent scale / instead of: converting them too.
  Confidence: high. Neither resolves to a registry entry, so they are published namespaced and there is no canonical scale to convert onto. They now carry the point the rewritten tests were making about per-harness bounds tables.
  General? no.

Coverage: 22 declared-bound sites found by sweep, 22 fixed. 12 canonical metric mappings audited against the registry, 1 conflict found, 1 fixed. 0 dropped.

Operator asked about policy calls? No new canonical id, no schema or validator rule change, no data dropped. The two judgement calls are the AA indices and the registry entry, both above.

Not in this PR

• make_evaluation_result never supplies the source_data that EvaluationResult requires, so it cannot build a result at all. Same module, separate defect.
• bfcl, cocoabench and sciarena have no adapter test of their own, so three of the five changed adapters are covered only by the shared publication test.
• _aggregate_check_score_metadata never looks at uncertainty, so a rescaled score can still keep an unrescaled spread and pass the gate. That check needs the failing count over datastore main first.

[borgr]

Rebased onto main, which now carries #190. The one conflict was in converters/alpaca_eval/adapter.py, and it resolved to nothing: #190 rewrote _evaluation_results to loop over metric specs, and its _AVG_LENGTH_METRIC already declares fallback_max=float('inf') with the same reasoning this PR's commit gave. So the alpaca_eval part of "Stop declaring an observed maximum as a metric's ceiling" is already in main and the commit now only touches the five adapters — arc_agi, artificial_analysis, bfcl, cocoabench and sciarena.

All five checks green.