OpenAI Launches Decisions API for Fast, Cheap AI Choices — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- OpenAI made its Decisions API widely available to all customers, reconfiguring ChatGPT Luna to deliver responses in 100–180 milliseconds, down from 1–3 seconds for standard LLMs.
- Nikunj Handa, OpenAI’s API product lead, said the team built the Decisions API rapidly after internal recognition of a 'million use cases' sparked by TypeSafe’s Jev model.
- TypeSafe’s Jev model established the decision model category, enabling fast, low-cost classification tasks and helping the startup reach a $7.6 billion valuation.
- Box, Datadog, Ramp, and Speak tested the Decisions API, with Speak cutting response latency from 1.4 seconds to 0.13 seconds in its AI language tutoring platform.
- Public benchmarks like the AIM System One Index and Artificial Analysis Decision Index rank Liquid’s D1 Ultra and Base Lab’s Elephant-1 ahead of OpenAI’s model, which placed eighth and fourteenth respectively.
- JevMan, a PacMan-playing benchmark created by Opper AI, shows decision models excelling under rapid-fire choices, with the latest Jev model topping the leaderboard.
- OpenAI added voice and visual modalities to the Decisions API and optimized global latency by targeting inference resources geographically, aiming for future sub-60ms performance.
Why it matters: Software developers now have a viable, low-latency alternative to full LLMs for decision-heavy workflows, reducing costs and unlocking real-time applications. OpenAI’s entry validates the decision model market, but lags behind leaders in benchmarks despite strong multi-modal potential.
Ask SkimNews
