fix(kokoro): COLA-normalize iSTFT deconv weights (FluidAudio #852) - #83
Open
Alex-Wengg wants to merge 1 commit into
Open
fix(kokoro): COLA-normalize iSTFT deconv weights (FluidAudio #852)#83Alex-Wengg wants to merge 1 commit into
Alex-Wengg wants to merge 1 commit into
Conversation
CoreMLCustomSTFT replaces torch.istft with ConvTranspose1d overlap-add but omitted the summed-squared-window (COLA) normalization torch.istft applies, leaving output exactly 1.5x too loud (periodic Hann, win 20, hop 5 -> interior envelope sum(w^2) = 1.5, a pure scalar; edge taps are sliced off by the center pad). Fold the envelope into the synthesis weights in both the laishere (en/ja) and v1.1-zh conversion scripts. This reconstructs the fix described in FluidAudio PR #699, which was built and measured (1.02x PyTorch raw level, jf_alpha peak 0.306 vs 0.299) but never committed; only the built artifact survived in build/tail-fix/. The reconstruction reproduces that artifact's deconv weights to within 1 ulp (max abs diff 7.5e-9, fp32 rounding order). Also commit convert-voices.py (laishere voice-pack extraction used for ANE-ja/, referenced by #699 but likewise never committed). Corrected tails are published as KokoroTail_v2.mlmodelc alongside the originals in ANE/, ANE-ja/, ANE-zh/ of FluidInference/kokoro-82m-coreml (commit acac8811); consumer adoption is tracked in FluidAudio #852.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FluidInference/FluidAudio#852 reports that the COLA-corrected
KokoroTaildescribed in FluidAudio PR #699 never made it into the published bundles — and it turns out the script fix never made it into mobius either. Only the built artifact survived locally (build/tail-fix/, Jun 13). This PR reconstructs and commits that fix.CoreMLCustomSTFTreplacestorch.istftwith aConvTranspose1doverlap-add but omitted the summed-squared-window (COLA) normalizationtorch.istftapplies. Analysis and synthesis each apply the periodic Hann window once, so the overlap-added output carries asum(w^2)envelope — a constant 1.5 in the interior for win 20 / hop 5 (edge taps are sliced off by the center pad). Result: raw output exactly 1.5× the PyTorch reference, pure scalar, no spectral change.The fix folds the envelope into the synthesis deconv weights, in both the laishere (en/ja) and v1.1-zh conversion scripts, with an assert that the interior envelope is actually constant.
Also commits
convert-voices.py(laishere voice-pack extraction used to buildANE-ja/voices, referenced by FluidAudio #699 but likewise never committed).Validation
build/tail-fix/artifact (the one #699 measured at 1.02× PyTorch, jf_alpha peak 0.306 vs 0.299) to within 1 ulp: max abs diff 7.5e-9, pure fp32 rounding order.Published artifacts
Corrected tails are live as
KokoroTail_v2.mlmodelcalongside the originals inANE/,ANE-ja/,ANE-zh/ofFluidInference/kokoro-82m-coreml(commit acac8811); originals kept so existing consumers are unaffected until they opt in. Swift-side adoption is tracked in FluidInference/FluidAudio#852.🤖 Generated with Claude Code