polarquant

AI agent skill implementing Google's TurboQuant compression algorithm (ICLR 2026) — 6x KV cache memory reduction, 8x speedup, zero accuracy loss. Compatible with Claude Code, Codex CLI, and all Agent Skills-compatible tools.

Near-optimal vector quantization for LLM KV cache compression. Python implementation of TurboQuant (ICLR 2026) — PolarQuant + QJL for 3-bit quantization with minimal accuracy loss and up to 8x memory reduction.

Drop-in KV cache compression for MLX on Apple Silicon. Brings PolarQuant (Google, ICLR 2026) to mlx-lm with first-class Gemma 4 support: MatFormer, dual head_dim, hybrid sliding/global attention, cross-layer KV sharing. 3-bit → 4.8× smaller cache, 0.995 logit cosine @ 4-bit.

Compress LLM KV cache with PolarQuant K+V quantization for 73-99% VRAM savings on consumer GPUs with zero token loss

Local AI agent with 16K context on 8GB RAM. No cloud, no API keys.

Universal vector compression — from embeddings to KV cache.

Build a local offline AI agent for Apple Silicon with 16K context, 8GB RAM support, and no cloud, API keys, or subscriptions

High-performance LLM Quantization & KV Cache Compression Engine. Featuring PolarQuant technology, universal model patching, and 4-bit KV caching for extreme memory efficiency.

Reproduction of TurboQuant.

A Rust implementation of TurboQuant - Google Research's near-lossless KV-cache compression algorithm for large language models (to be presented at ICLR 2026)