firered: evaluate the ONNX punctuator, and whether torch-firered can go

[fyang0507]

Why this is worth asking

torch-firered exists for one reason: FireRed pins transformers==5.1.0, which nothing else
here accepts. It cannot move to mlx because mlx-audio 0.4.5 implements FireRed's AED only
— Conformer encoder, transformer decoder, beam search — and ships no punctuation restoration
model anywhere (every punc hit in the package is TTS text normalization). Punctuated text is a
floor, so a FireRed stack without FireRedPunc is not a conforming backend, and moving only
the AED would split one stack across two environments.

But the auxiliary models have third-party conversions, and core already carries onnxruntime
for Silero VAD. If the punctuator runs there, the reason torch-firered exists gets much
weaker — and with VAD and LID as well, it could disappear, taking PyTorch out of the tool
entirely.

Evidence for the blocker: model_tests/benchmark/results/2026-08-17-mlx-collapse-probes.json,
firered section. See also #11.

Candidates, none evaluated

Role	Torch (measured)	Conversion found	Where it could run
punctuator (floor)	FireRedPunc	42ailab/FireRedPunc-ONNX	core — no new environment
lid	FireRedLID	aufklarer/FireRedLID-887M-MLX-8bit, cstr/firered-lid-GGUF	mlx (needs code; mlx-audio has no FireRed LID module)
vad	FireRedVAD	tardigrade-doc/FireRedVAD_onnx, illitan/FireRedVAD-CoreML	core, or drop to silero-vad and lose vad: native
asr	FireRedASR2-AED	mlx-community/FireRedASR2-AED-mlx (4.25 GiB, license: none declared)	mlx

All third-party, none run here, none with measured equivalence to the recorded output. The
process rule applies with force: these are repository names, not evidence.

Sequence, cheapest and most decisive first

1. The punctuator. It is the blocker, it is the smallest model (1.21 GiB torch), and ONNX in
   core needs no new environment. Test: run it over the recorded FireRed artifacts' sentence
   text and compare against the recorded punctuated output. The bar is not "produces
   punctuation" — it is the punctuation-floor invariant, that stripping punctuation and
   whitespace from a sentence's text yields the concatenation of its word texts compared
   case-insensitively, plus the recasing behaviour (FireRedPunc lowercases then re-capitalizes
   sentence starts and standalone i; 234 characters differ by construction across the recorded
   artifacts). A conversion that drops the recasing changes the invariant's shape.
2. The AED, only if the punctuator works. Compare text against the five recorded FireRed
   artifacts (12,370 word tokens) and check that word timing survives conversion — FireRed is the
   only stack with native subtitle-grade timing, and it builds its word stream from
   pre-punctuation AED timestamps (fireredasr2system.py:181-184). If the MLX conversion loses
   that, this stack has no reason to move at all.
3. LID and VAD, last and separable. lid is a requestable capability that only FireRed
   satisfies, so losing it is a spec change; native vad likewise. Either could stay in
   torch-firered if only that stage needs it — but then the environment survives and the
   exercise was pointless, so decide up front that partial is not interesting here.

Acceptance

• Each conversion either measured against the recorded artifact or explicitly declined, with the
  reason recorded. "Looks equivalent" is not a result.
• If all four move: delete requirements/torch-firered.in and its lock, and note in
  ENVIRONMENTS.md that no PyTorch environment remains — that would leave
  mlx, core, and swift, and make the transformers conflict that drove the whole layout
  historical.
• If they do not: record which one blocked it, so nobody re-runs this from scratch.
• Licenses: mlx-community/FireRedASR2-AED-mlx declares none, against Apache-2.0 upstream.
  license_declared: null with license_reviewed: false is the honest entry, and it is worth
  knowing before adopting a conversion of an Apache-2.0 model that dropped the notice.