Cua Metal Shim: macOS VM Llama.cpp Up to 16× Faster
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Cua released a Metal capability shim as a research artifact that intercepts selected capability queries inside one guest process, returning Apple family 9 and 64 KB threadgroup memory instead of the stock paravirtualized profile (Apple 5-era family, 32 KB threadgroup memory) so llama.cpp selects newer SIMD-group, SIMD-group matrix, and bfloat16 paths.
- On an Apple M1 Ultra (48-core GPU) running macOS 26.6.1 with a Tahoe 26.5.2 guest in Lume 0.5.1, TinyLlama 1.1B Chat Q4_K_M produced 11.08× faster prompt processing and 16.36× faster token generation than the stock VM, reaching 98% of bare-metal prompt speed and 72.06% of bare-metal generation speed.
- Google's Gemma 4 12B QAT Q4_0 GGUF improved 7.20× on prompt processing and 14.54× on token generation with the unlocked VM, hitting 99.59% of bare-metal prompt speed and 94.82% of bare-metal generation speed across ten llama-bench samples.
- Meta's official Muse Glimmer 30B Q4_K-M GGUF (16.76 GB) tested in a 64 GiB Tahoe guest with llama.cpp b10359 ran 7.55× faster on a 512-token prompt and 8.87× faster on 128-token generation, in a text-only configuration without Ollama, multimodal projectors, or drafters.
- MLX-LM 0.31.3 with Llama-3.2-3B-Instruct-4bit stayed flat because MLX was already fast in the stock VM, and an ablation that advertised MTLGPUFamilyMetal3 caused MLX to request an unavailable residency set, so the release shim intentionally leaves Metal 3 at its stock value.
- Apple's Virtualization.framework uses paravirtualization rather than VFIO-style GPU passthrough; Cua notes the shim relies on private, version-sensitive behavior in the guest's Metal implementation that Apple may change between macOS releases, requiring each host/guest combination to be retested independently.
Why it matters: For developers running llama.cpp workloads inside macOS VMs on Apple Silicon, this shim narrows the virtualization penalty to roughly 0.4–5% on prompt processing against bare metal across three tested models (TinyLlama, Gemma 4, Muse Glimmer), making Cua's Lume-based local computer-use environments substantially more viable for LLM-driven agents. The trade-off is supportability: the technique targets private Metal behavior Apple has not documented as intended for VMs, so results could regress with future macOS releases until Apple clarifies the policy.
Ask SkimNews
