Cactus Releases Needle 2: 45M-Param LLM in 14MB
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Cactus released Needle 2, an open 45M-parameter model for tool calling, device use, and structured extraction, shipped as one 14MB binary with a deterministic 28MB session RAM ceiling, licensed Apache 2.0 with weights on Hugging Face.
- Needle 2 trades wins on Google's Mobile Actions, DroidCall, Seal-Tools, and BFCL v4 against FunctionGemma 270M, LFM2.5 230M, and Apple FM at 5×–70× smaller and at CQ2-bit precision versus their f16.
- Pebble runs Needle 2 locally in its Index 01 app on the screenless Index Ring, turning spoken requests into device actions with no network dependency.
- Needle 2 was pretrained on 115B tokens and post-trained on 38B — roughly 120× less than LFM2.5-230M's 19 trillion pretraining tokens — with weights, activations, and KV cache all quantized to CQ2-bit throughout training rather than post-hoc.
- The model uses a 256-token sliding-window KV cache that pins system prompt and tool declarations as permanent sinks, runs on ESP32-S3-class microcontrollers, and ships as a dependency-free C++ binary that self-selects CPU kernels (NEON, AVX2, SDOT, RISC-V vectors, wasm SIMD, or scalar) at startup.
- Decoding hits 500 tokens/sec on Raspberry Pi 5, 400–1,500 on Meta Quest 3S and Apple Vision Pro, and 300–700 on sub-$200 Samsung A-Series phones, at 7×–85× fewer MFLOPs per token than benchmarked peers.
Why it matters: Cactus is betting the real edge-AI market is the ~21 billion connected IoT devices and sub-$200 phones — not the 1.5 billion PCs that dominate 'edge AI' marketing — and that 45M params at 14MB with 7–85× fewer MFLOPs per token is enough for tool calling if the model admits what it doesn't know via a confidence score.
Ask SkimNews
