norm-separation

Here are 2 public repositories matching this topic...

metaSATOKEN / nsn

Training-free INT3 KV cache quantization: 5.09× compression, ~10 lines of Python, <5% WikiText-2 ΔPPL on 8 of 8 open-weight Transformers (GPT-J 2021 → Gemma-4 2026). No calibration, no codebook, no rotation, no adapter. +2.4% decode overhead with torch.compile (no custom CUDA).

reproducible-research transformers pytorch open-science quantization memory-optimization kv-cache int4 calibration-free llm-inference int3 norm-separation per-channel-quantization activation-outliers

Updated May 21, 2026
Python

metaSATOKEN / norm-separated-quantization

Star

Training-free fix for KV cache INT4 failures. Norm separation + per-channel quantization. Qwen2-7B: 744× improvement (ΔPPL +238 → +0.32). 12 models, 124M–40B. 4 lines of PyTorch.

reproducible-research pytorch open-science transformer outliers quantization memory-optimization kv-cache int4 llm-inference norm-separation per-channel-quantization

Updated Apr 18, 2026
Python

Add this topic to your repo

To associate your repository with the norm-separation topic, visit your repo's landing page and select "manage topics."

Learn more