ZML launches free multi-chip AI inference server
SkimNews Take
By offering free, chip-agnostic inference software, ZML commoditizes the software layer that has historically differentiated each chip vendor's ecosystem — pushing competition down to silicon-level economics like memory bandwidth and interconnect density.
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- ZML launched ZML/LLMD, a free inference server that runs open-source LLMs across Nvidia, AMD, Google TPU, Apple Metal, and Intel Arc chips, with founder Steeve Morin telling TechCrunch the goal is to break vendor silos and run each chip at maximum speed.
- Steeve Morin, ZML's Paris-based founder and former Zenly VP of engineering (which Snapchat acquired for nine figures in 2017), said his 20-person team is already "co-designing silicon" with European chipmakers including Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA.
- ZML raised $20 million from VCs including Harry Stebbings' 20VC, Xavier Niel's Kima Ventures, Kindred Capital, LocalGlobe, Puzzle Ventures, >commit, AALVC, and Drysdale Ventures; its cap table also includes Yann LeCun, Solomon Hykes, and Hugging Face's Clément Delangue and Julien Chaumond.
- Competitors in what the industry calls the "inference gold rush" include Baseten, recently valued at $13 billion, Inferact from the creators of vLLM, and RadixArk, the commercial company behind SGLang.
- Unlike ZML's 2024 open-source ML framework, ZML/LLMD is free but not open source; Morin said he chose a proprietary free tier to "measure and then generate revenue where it is most effective without hindering my growth stupidly."
Why it matters: With Baseten already valued at $13 billion in the inference layer, ZML's free, chip-agnostic software is a direct play for enterprises and clouds that want to mix cheaper or more energy-efficient silicon into AI workloads instead of paying full freight to Nvidia — and a 20-person Paris team co-designing silicon with European chipmakers signals Europe's hardware ambitions now have a matching software stack.
