Google Shrinks Gemma 4 E2B to Under 1GB With QAT
Get the Sports newsletter
Daily sports — scores, transfers, the storylines from the leagues you actually follow. Free.
- Google released Quantization-Aware Training (QAT) checkpoints for Gemma 4, integrating quantization into training rather than applying it afterward, which the company says preserves more quality than standard Post-Training Quantization.
- The QAT release includes checkpoints for the Q4_0 format plus a novel mobile-specialized schema that compresses Gemma 4 E2B to under 1GB of memory, enabling on-device inference on everyday edge hardware.
- The mobile schema uses static activations, channel-wise quantization aligned to mobile accelerators, targeted 2-bit quantization for token-generation layers, and optimized embeddings plus KV cache to shrink active memory.
- Gemma 4 E2B text-only (without audio and vision encoders) requires less than 1GB, letting users trim the footprint further by deploying only the modalities they need.
- Google partnered with Hugging Face, llama.cpp, vLLM, Ollama, LM Studio, LiteRT-LM, Transformers.js, SGLang, MLX and Unsloth to support the new QAT checkpoints across desktop, web, and on-device workflows starting at launch.
- The QAT release follows Multi-Token Prediction support and a 12B model added in the two months since Gemma 4's initial release, positioning QAT as the next step in a rapid compression push.
Why it matters: Running a capable language model locally on a phone under 1GB removes the latency, cost, and privacy tradeoffs of cloud inference for developers building on-device AI features. Google is making Gemma 4 directly competitive in the mobile-LLM space where size, not just benchmark scores, determines what actually ships in apps.
