Enable direct GLM-5.2 GGUF tensor mapping - #682
Open
justinchuby wants to merge 2 commits into
Open
Conversation
justinchuby
force-pushed
the
tyrell/glm52-real-preflight
branch
from
August 27, 2026 21:37
c34b519 to
93e0fc6
Compare
Promote glm-dsa GGUF import through exact config restoration, MLA/DSA tensor mappings, and split K/V-B projection handling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
justinchuby
force-pushed
the
tyrell/glm52-real-preflight
branch
from
August 27, 2026 22:09
93e0fc6 to
5163cdc
Compare
Performance Comparison
|
🏗️ Architecture Diff
No architecture changes detected. ✅ Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 39ff6824-d35f-4d3b-8f5f-043a7119a100
Member
Author
|
Independent revision evidence (Deckard; original author was not consulted):
Validation:
The 216.7 GB checkpoint was not rerun or copied. Please assign an independent reviewer other than the original author and this revision owner. Do not merge pending that review and CI. |
Member
Author
|
Exact-head CI follow-up for |
Contributor
There was a problem hiding this comment.
Pull request overview
Enables direct GGUF tensor-name mapping and tensor-shape restoration for the GLM-5.2 glm-dsa architecture so GGUF weights can be routed into the existing glm_moe_dsa graph (without whole-model dequantization), while keeping the routed DSA/MLA MTP block fail-closed.
Changes:
- Split MLA KV-B projection handling for GLM-5.2 by emitting distinct
k_b_proj/v_b_projparameters in the DSA attention path and adding apreprocess_weights()shim to split HF’s fusedkv_b_proj. - Add
glm-dsaGGUF tensor mapping + tensor processor to restore GGUF’s rank-3 K/V-B layouts to the graph’s expected linear row-major form, and wire this through the GGUF architecture registry. - Update/expand tests and documentation to reflect GLM-5.2’s official config geometry and the newly importable architecture count.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/mobius/models/glm_moe_dsa.py | Emit split KV-B projections in GLM DSA attention and split HF fused KV-B weights during preprocessing. |
| src/mobius/models/glm_moe_dsa_test.py | Add coverage ensuring preprocess_weights() covers all graph parameters and validates KV-B splitting. |
| src/mobius/integrations/gguf/_tensor_processors.py | Add glm_dsa tensor processor to reshape split MLA K/V-B tensors into linear row order. |
| src/mobius/integrations/gguf/_tensor_processors_test.py | Add unit test validating GLM-5.2 MLA K/V-B layout restoration. |
| src/mobius/integrations/gguf/_tensor_mapping.py | Add GGUF→HF tensor-name mapping recipe for glm-dsa. |
| src/mobius/integrations/gguf/_route_census_test.py | Update pinned route census expectations for architecture:glm-dsa. |
| src/mobius/integrations/gguf/_glm_moe_dsa_test.py | Pin official GLM-5.2 geometry/config expectations and validate tensor-name mappings + config rejection cases. |
| src/mobius/integrations/gguf/_config_mapping.py | Restore official GLM-5.2 config fields/constraints in GGUF postprocessing and enforce SIGMOID gating requirement. |
| src/mobius/integrations/gguf/_builder.py | Treat glm-dsa MLA K/V-B projections as reshaped tensors (special-casing native-block replacement and quant import routing). |
| src/mobius/integrations/gguf/_arch_registry.py | Promote glm-dsa from deferred tensor mapping to supported tensor mapping + processor + postprocessor wiring. |
| src/mobius/integrations/gguf/_arch_registry_test.py | Update expected supported architecture count and include glm-dsa in the supported set. |
| src/mobius/components/_deepseek_mla.py | Add optional split-KV-B projection construction/forward path for MLA modules. |
| docs/api/build_from_gguf.md | Update support-matrix counts and move glm-dsa into the importable set while keeping runtime deferred. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
justinchuby
added a commit
that referenced
this pull request
Aug 28, 2026
## Summary - add exact explicit-float MiniMax-M2 import with non-square Q/O geometry, full-vector Q/K RMSNorm, partial NeoX RoPE, F32 correction-biased sigmoid routing, and dynamic/static KV coverage - add Mistral4 as its pinned DeepSeek-V2 MLA/MoE architecture, including a dedicated latent K-only cache and metadata-owned YaRN scaling; correct the vendored llama.cpp census that previously mislabeled it as Mistral3 - complete GLM-DSA tensor routing and split K/V-B transforms while preserving the dedicated DSA cache contract and fail-closed MTP boundary - promote all three graph routes in the registry/census and regenerate the GGUF support documentation This includes and supersedes the draft GLM-only work in #682. ## Post-review fixes - canonicalize `glm_dsa` aliases for validation/task/package dispatch while retaining the raw alias prefix for metadata reads; the alias-spelled GGUF proves the exact mixed full/shared indexer schedule, dedicated default task, complete indexer initializer materialization, and executable dense-MLA logits - consume Mistral4's serialized `rope.scaling.yarn_log_multiplier` through `yarn_apply_mscale`; an independent metadata oracle pins `0.0707`, factor `4`, qk dim `4` to `0.6028141`, and mutation to `0.1` discriminates the result - fail closed during Mistral4 config extraction unless the multiplier is a non-boolean finite number in the inclusive semantic range `[0.0, 0.1]`; the wire endpoint is `float(np.float32(0.1))`, used for both validation and normalization - reject NaN, `+inf`, `-inf`, negative values, and the immediate float32 successor above `0.1`; real `GGUFWriter`/`GGUFModel` round trips accept both endpoints and prove finite attention scale and rotary caches before graph execution - call the immediate Mistral4 superclass initializer and force `GlmMoeDsaTask()` when callers omit `task` - reply to and resolve both GitHub review threads ## Evidence and capability boundaries Pinned semantic source: `ggml-org/llama.cpp@8d9af256337d1a501250f9bbf4c0859a654bddd6`. All three real candidates exceed the 16 GiB bounded-artifact policy, so the PR records immutable metadata-only LFS evidence and uses reduced synthetic GGUFs for executable graph/cache parity: | Architecture | Smallest pinned candidate | Complete bytes | |---|---|---:| | `minimax-m2` | `mradermacher/MiniMax-M2-i1-GGUF@2d4f9b1a86d32ce4dfc47db312c8d6fcae8d7b37` / `MiniMax-M2.i1-IQ1_S.gguf` | 46,514,882,176 | | `mistral4` | `unsloth/Mistral-Small-4-119B-2603-GGUF@bd93c721735aa32c035c0f19e738cb3371fd56ff` / `Mistral-Small-4-119B-2603-UD-IQ1_M.gguf` | 32,306,941,632 | | `glm-dsa` | `unsloth/GLM-5.2-GGUF@abc55e72527792c6e77069c99b4cb7de16fa9f23` / six `UD-IQ1_S` shards | 216,715,360,960 | No GGUF payload was downloaded. Hub probes read file metadata only. Runtime support remains `DEFERRED`, and quantization-preserving import is `REJECTED`; callers must use `keep_quantized=False`. GLM routed DSA/MLA MTP, Mistral4 NextN/legacy MLA/Q-LoRA-free/nonzero-temperature layouts, and unowned GLM RoPE scaling all fail closed. There are no production model-ID or revision gates. **Coverage waivers:** real-weight L2/L4/L5 and runtime generation evidence cannot fit the artifact budget. Synthetic tests instead execute full-logit prefill/cached decode, MiniMax static cache, Mistral latent-vs-expanded MLA parity, GLM indexer scheduling, alias-spelled weight closure, and weight application. ## Rebase reconciliation Rebased onto #685 at `86e70a846b572a5266170420373781d2cac23450`. The combined tree preserves the Grok/GroveMoE/Hunyuan-MoE and MiniMax-M2/Mistral4/GLM-DSA cohorts in registry dispatch, tensor-contract validation, architecture fingerprint isolation, model coverage, parity exclusions, census, and generated docs. Combined pins are 105 supported architectures and `107` dependency-blocked / `100` evidence-only / `47` immediately implementable remaining routes. ## Validation - focused combined GGUF/model coverage suite: `2417 passed, 273 skipped` - immutable Hub metadata probes: `3 passed` - broad serial non-integration suite: `8981 passed, 64 skipped, 12 deselected, 1 subtest passed` - `lintrunner f --output oneline --all-files` and `lintrunner -a --output oneline` - `python scripts/generate_gguf_support_docs.py --check` - `git diff --check origin/main...HEAD` - changed-line mypy: `0` errors (`29` pre-existing production-file errors outside changed lines) - independent max-effort review found no significant issues and verified both cohorts, the float32 YaRN endpoint, pre-graph failure timing, fingerprint isolation, tensor mapping, cache semantics, census/docs, and fail-closed ordering Base: `86e70a846b572a5266170420373781d2cac23450` Head: `208f2cb3c7c5215ea9bcf8547a54109acf768bee` --------- Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0ee8718a-33b9-4015-9e20-3745bad54686
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
glm-dsafrom deferred tensor mapping to the existingglm_moe_dsagraphHEAD:
5163cdc2082bad63a2ec87f41e9400890ea93443Official checkpoint evidence
Source:
UD-IQ1_S, 6 shards, exactly 216,715,360,960 bytes. Header-only validation found 1,809 tensors and 0 unmapped base-decoder tensors; the 21 remaining tensors are the appended routed DSA/MLA MTP block. No checkpoint bytes were copied.Exact projection inventory:
Q5_K, downQ6_KIQ1_S: layers 3-5, 7, 10-28, 30-38, 49-67, 69, 71IQ2_XXS: layers 6, 8-9, 29, 39-48, 68, 70, 72-77IQ3_XXS: layers 3-7, 9-74;IQ4_XS: layer 8 and 75-77Q5_K: layers 3-7, 9-78; layer 8 isQ6_KQ6_K: layers 3-7, 9-78; layer 8 isQ8_0F32Q2_K, routed downQ3_KReal streaming attempt
env -u MOBIUS_ALLOW_DENSE_MOE_EXPERTS -u MOBIUS_ENABLE_BQMOE_PERPROJ_V2 \ PYTHONPATH=src PYTHONUNBUFFERED=1 python3 -c \ "from mobius.integrations.gguf import build_from_gguf; build_from_gguf('/datadisks/disk5/justinchu/onnx-genai-models/unsloth-GLM-5.2-GGUF/checkpoint/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf', keep_quantized=True, execution_provider='default', allow_dense_moe=False)"No-dense-fallback evidence is explicit in the command: the opt-in environment variable is unset and
allow_dense_moe=False. Routed IQ expert bytes stay on the native quantized MoE path.First deterministic blocker:
Responsible API:
mobius.integrations.gguf._mtp.validate_mtp_tensor_contract. The next slice is a dedicated GLM routed DSA/MLA MTP sidecar; this PR does not bypass, truncate, densely lower, or redesign it and does not touch the native CUDA IQ consumer.Bytes and files
216,715,360,960bytes205,391,926,272bytes/datadisks/disk5free-space accounting: preflight delta 0; final attempt sample changed by 16,384 bytes while the checkpoint remained exactly the same six files and no export artifact appearedValidation
1567 passed, 1 skippedacross registry, config mapping, tensor mapping/processors, GLM integration/model, and MTP testsgit diff --checkpassed