Skip to content

Split GGUF builder tests into focused modules - #705

Open
justinchuby wants to merge 1 commit into
mainfrom
justinchuby-split-gguf-builder-tests
Open

Split GGUF builder tests into focused modules#705
justinchuby wants to merge 1 commit into
mainfrom
justinchuby-split-gguf-builder-tests

Conversation

@justinchuby

Copy link
Copy Markdown
Member

Summary

  • extract synthetic GGUF writers and reusable fixtures into private, non-collected _builder_test_utils.py
  • partition the 451 builder tests into core quantization/reuse, architecture cohorts, and contracts/preflight modules
  • update cross-test imports, documentation, and quantization evidence references; remove the original collected module

Validation

  • old/new collection: 451 / 451, with identical normalized class/function/parameter node IDs
  • all 53 top-level test/helper definitions are AST-equivalent after relocation
  • 490 passed across all replacement modules, _dense_c01_test.py, and the affected MTP cross-import test
  • 92 passed for affected-model discovery and GGUF evidence/docs regressions (plus 20-pass confirmation after evidence path correction)
  • lintrunner passed on all changed files
  • git diff --check passed
  • independent code review found no missing tests, duplicate collection, import cycles, or behavior changes

Extract shared synthetic GGUF writers and fixtures into a private support module, partition the collected tests by responsibility, and update evidence and cross-test references without changing test behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
@justinchuby
justinchuby requested review from a team and a lite review from Copilot August 30, 2026 15:30
@github-actions

Copy link
Copy Markdown

🏗️ Architecture Diff

Comparing dda71aa69c2966

Model Sub-model Changes Status

No architecture changes detected.


Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed)

@github-actions

Copy link
Copy Markdown

Performance Comparison

Comparing dda71aa69c2966

Model Metric Baseline Current Delta
bert (feature-extraction) model_size_bytes 359 KB 359 KB +0.0%
bert (feature-extraction) num_nodes 68 68 +0.0%
falcon model_size_bytes 364 KB 364 KB +0.0%
falcon num_nodes 66 66 +0.0%
gemma2 model_size_bytes 428 KB 428 KB +0.0%
gemma2 num_nodes 105 105 +0.0%
gpt2 model_size_bytes 324 KB 324 KB +0.0%
gpt2 num_nodes 54 54 +0.0%
llama model_size_bytes 425 KB 425 KB +0.0%
llama num_nodes 60 60 +0.0%
llama (static-cache) model_size_bytes 425 KB 425 KB +0.0%
llama (static-cache) num_nodes 56 56 +0.0%
mamba (ssm-text-generation) model_size_bytes 296 KB 296 KB +0.0%
mamba (ssm-text-generation) num_nodes 94 94 +0.0%
phi3 model_size_bytes 421 KB 421 KB +0.0%
phi3 num_nodes 58 58 +0.0%
phi3 (static-cache) model_size_bytes 421 KB 421 KB +0.0%
phi3 (static-cache) num_nodes 54 54 +0.0%
qwen2 model_size_bytes 425 KB 425 KB +0.0%
qwen2 num_nodes 60 60 +0.0%
qwen2 (static-cache) model_size_bytes 425 KB 425 KB +0.0%
qwen2 (static-cache) num_nodes 56 56 +0.0%
qwen3_5_moe (hybrid-text-generation) model_size_bytes 506 KB 506 KB +0.0%
qwen3_5_moe (hybrid-text-generation) num_nodes 265 265 +0.0%
qwen3_5_text (hybrid-text-generation) model_size_bytes 458 KB 458 KB +0.0%
qwen3_5_text (hybrid-text-generation) num_nodes 127 127 +0.0%
qwen3_5_vl (hybrid-qwen-vl) model_size_bytes 977 KB 977 KB +0.0%
qwen3_5_vl (hybrid-qwen-vl) num_nodes 450 450 +0.0%
t5 (seq2seq) model_size_bytes 836 KB 836 KB +0.0%
t5 (seq2seq) num_nodes 176 176 +0.0%
whisper (speech-to-text) model_size_bytes 1008 KB 1008 KB +0.0%
whisper (speech-to-text) num_nodes 128 128 +0.0%

No performance regressions.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The updated test-infrastructure map currently omits still-present key GGUF integration test modules, making the documentation misleading.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR refactors the GGUF builder test suite by splitting the former monolithic builder test module into multiple focused test modules and extracting shared synthetic GGUF writers/fixtures into a private, non-collected utility module. It also updates cross-test imports plus references in quantization evidence and documentation so internal “evidence → test” links remain accurate.

Changes:

  • Added src/mobius/integrations/gguf/_builder_test_utils.py to centralize synthetic GGUF writers and shared fixtures for builder tests.
  • Split builder tests into focused modules (core/architectures/contracts) and updated dependent tests/import sites accordingly.
  • Updated quantization evidence references and docs to point at the new test module locations.
File summaries
File Description
testdata/evidence/gguf_quantization_capabilities.json Updates evidence-to-test node IDs to the new builder test module paths.
src/mobius/integrations/gguf/_quant_capabilities.py Updates hard-coded test references for quantization capability evidence.
src/mobius/integrations/gguf/_mtp_test.py Switches from importing the removed _builder_test to using _builder_test_utils.
src/mobius/integrations/gguf/_mmproj_test.py Updates docstring reference to the former builder test module.
src/mobius/integrations/gguf/_dense_c01_test.py Switches helper import to _builder_test_utils.
src/mobius/integrations/gguf/_builder_test_utils.py New private helper module containing synthetic GGUF writers and shared fixtures.
src/mobius/integrations/gguf/_builder_contracts_test.py New focused test module for builder preflight/cache/tensor/weight contracts.
src/mobius/integrations/gguf/_builder_core_test.py New focused test module for core builder/quantization/reuse behaviors.
src/mobius/integrations/gguf/_builder_architectures_test.py New focused test module for architecture-cohort builder behaviors.
src/mobius/integrations/gguf/_builder_test.py (deleted) Removes the original monolithic collected builder test module after splitting.
docs/design/test-infrastructure-map.md Updates documentation references for the GGUF test suite layout.
Review details
  • Files reviewed: 8/11 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

| **Components** | 67 | 379 | 23 test files (rotary_embedding 28M, quantized_linear 22M, audio 16M, attention 17M, vision 16M, whisper 20M, etc.) |
| **Weight Handling** | 17 | 114 | `_weight_utils_test.py` (58M), `_weight_loading_test.py` (27M), `_graph_diff_test.py` (24M) |
| **GGUF Integration** | 19 | 91 | `_builder_test.py`, `_reader_test.py` (26M), `_repacker_test.py` (30M), `_tensor_mapping_test.py` (18M) |
| **GGUF Integration** | 19 | 91 | `_builder_core_test.py`, `_builder_architectures_test.py`, `_builder_contracts_test.py`, `_reader_test.py` (26M) |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants