Apple Silicon VM Llama.cpp Hits 11–16× via Metal Shim
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Cua released a process-scoped Metal capability shim as a research build under the same permissive license as Lume, answering Apple-family queries up to family 9 (1009) and raising reported threadgroup memory from 32 KB to 64 KB so llama.cpp selects newer SIMD-group, matrix, and bfloat16 kernels.
- TinyLlama 1.1B running through llama.cpp on an M1 Ultra processed prompts 11.08× faster and generated tokens 16.36× faster inside an unlocked Lume VM than the same stock VM, reaching 98% of bare-metal prompt speed.
- Google's Gemma 4 12B QAT Q4_0 saw 7.20× faster prompt processing and 14.54× faster token generation with the same shim, hitting 99.59% of bare-metal prompt throughput and 94.82% of bare-metal generation speed on the M1 Ultra.
- The shim runs entirely within Apple's paravirtualized Virtualization.framework graphics path — no physical GPU assignment, VFIO passthrough, or kernel changes — and the host, guest kernel, and other guest processes keep their stock configuration.
- MLX-LM 0.31.3 on MLX 0.32.0 showed no speedup because it was already fast in the stock VM; advertising MTLGPUFamilyMetal3 during ablation made MLX request a residency set unavailable through the paravirtualized device, which shaped the release profile.
- Cua tested on one M1 Ultra (48-core GPU, macOS 26.6.1 host) with a Tahoe 26.5.2 Cua guest (8 vCPU, 16 GiB) using the official llama.cpp b10167 release, and published image digests, binary hashes, JSON output, and verification scripts so others can reproduce.
Why it matters: Apple Silicon developers running local LLMs inside macOS VMs have been capped well below host speed because Apple's paravirtualized driver reports an older GPU profile by default — Cua's shim lifts that ceiling, hitting 98% of bare-metal prompt throughput on TinyLlama and 94.82% on Gemma 4 12B on the tested M1 Ultra. The gains explicitly vary by host GPU, guest version, and workload, and Apple retains the right to alter the private guest Metal behavior the technique exploits in future macOS releases.
Ask SkimNews
