feat(model-manager): support LyCORIS LoKr LoRAs for Krea-2 - #9424
feat(model-manager): support LyCORIS LoKr LoRAs for Krea-2#9424Balrog994 wants to merge 5 commits into
Conversation
Krea-2 LoRAs in LyCORIS LoKr format were rejected at install time and could not be used. LoKr is the default output format of ai-toolkit, so this affects a fair number of community adapters. Two gaps in the Krea-2 path: - `LoRA_LyCORIS_Krea2_Config` required a complete lora_A/B or lora_down/up pair. LoKr carries Kronecker factors instead, so the probe rejected it even though the Krea-2 signature modules matched. The explicit-base override went through the same helper, so forcing the base did not help either. - `_SUFFIX_TO_VALUE_KEY` did not map the lokr_* suffixes, so `_group_by_layer` would have dropped the weights and left each layer holding only its alpha. The rest of the stack already handles LoKr: `LoKRLayer` exists and `any_lora_layer_from_state_dict` routes to it on `lokr_w1` / `lokr_w1_a`. Only the Krea-2 wiring was missing, matching its (WIP) status in invoke-ai#9304. The Krea-2 signature-module requirement is unchanged, so a LoKr targeting only generic transformer blocks is still not claimed by this base.
Resolves the import conflict in test_krea2_lora_conversion_utils.py (both sides added imports) and reconciles this branch with invoke-ai#9449, which taught the Krea-2 converter the kohya flattened layout. The two changes interact. invoke-ai#9449 rewrites a flattened module only when every suffix on it is one `_group_by_layer` knows, and left LyCORIS algorithms verbatim because `lokr_*` was not such a suffix. This branch makes it one, so a kohya-flattened LoKr module now un-flattens and applies instead of degrading to the per-layer "Failed to find module" warning. The hazard invoke-ai#9449 guards against -- the grouper's blind rsplit(".", 2) cutting inside a dotted module path -- does not apply to a suffix the grouper recognises. So `test_kohya_lycoris_algorithm_keys_do_not_abort_the_load` now carries its expected layer per case: the LoKr parametrizations expect the converted module, `hada_*` and `diff` still expect the verbatim key. The property the test guards -- that a mixed-algorithm file never aborts the whole adapter -- holds either way. Added a test covering the kohya + LoKr path end to end.
|
Merged #9449 landed just after your last merge and touched the same four files, which is where the Your per-module gate converts a flattened module only when every suffix on it is one The hazard your comment guards against — the grouper's blind Concretely: two of the five parametrizations of If you'd rather this PR left the kohya path completely alone, the alternative is to give that gate
|
Summary
Krea-2 LoRAs in LyCORIS LoKr format can't be installed at all at the moment. The probe turns them away with:
I hit this with an adapter trained in ai-toolkit, which emits LoKr by default, so I'd guess a fair few of the Krea-2 LoRAs going around are in the same boat.
The Krea-2 probe only knows about
lora_A/lora_Bandlora_down/lora_up. LoKr carries Kronecker factors instead (lokr_w1/lokr_w2, or the factored_a/_bpairs plus an optionallokr_t2), so_has_complete_lora_pair()says no and the adapter is never identified. Setting the base by hand doesn't get you anywhere either, that path goes through the same helper.There's a second one further down:
_SUFFIX_TO_VALUE_KEYdoesn't list thelokr_*suffixes, so even past the probe_group_by_layer()would drop the weights and leave every layer holding nothing but its alpha.The rest was already there.
LoKRLayerexists,any_lora_layer_from_state_dict()reaches for it the moment it seeslokr_w1, and the Qwen-Image converter handles LoKr fine. Only the Krea-2 side was never wired up, which fits it being marked WIP in #9304.So this adds the two missing pieces: a
_has_lokr_layer()helper that both config gates consult, and thelokr_*suffixes in the converter's suffix map. I left the signature-module check alone, so a LoKr that only touches generic transformer blocks still won't be claimed as Krea-2.Related Issues / Discussions
Follows on from #9304.
QA Instructions
Take any ai-toolkit LoKr for Krea-2 (keys look like
diffusion_model.txtfusion.<block>.attn.wq.lokr_w1) and install it through the Model Manager. Before, it's rejected with the error above. After, it installs asbase=krea-2, format=lycorisand turns up in the LoRA picker.I tested with a 32-module adapter on the text-fusion stage against a GGUF-quantised Krea-2 Turbo. It converts to 32
LoKRLayers, every target key resolves to a realnn.LinearinKrea2Transformer2DModel, the reconstructed weight (4×4 ⊗ 640×640 = 2560×2560) matches the layer it patches, and generation comes out visibly different with the LoRA switched on.8 new tests, covering full and factored LoKr, the native→diffusers key renaming, and that a LoKr without the Krea-2 signature modules still gets turned away.
tests/backend/patches/andtests/backend/model_manager/pass (1030 passed, 133 skipped, 1 xfailed), as do ruff check and format. I only ran them on 3.12/Windows locally, so the rest of the matrix is on CI.Merge Plan
Nothing special, backend only.
Checklist
What's Newcopy (if doing a release after this PR)