GPT-6 Astra Drives a Toyota Through In-N-Out — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Aditya Ramabadran, Simon Mahns, and Tobias Gessler, AI engineers at startup Axiom, used OpenAI's GPT-6 Astra to drive a 2024 Toyota Corolla through an In-N-Out Burger drive-thru by routing a chat interface through a server to windscreen-mounted cameras and the car's power steering, with a safety driver keeping a foot above the brake.
- The trio's DrivingBench, which tests models on a simple parking lot course, found only GPT-6 Astra completed the loop (slowly), Anthropic's Claude Fable 5.1 made it 45% of the way, and SpaceXAI's Grok managed just 11%.
- Ramabadran says the models adapted to errors in real time through in-context learning, and the engineers believe the driving ability emerged as a side effect of training for 3D reasoning rather than any driving-specific instruction — "an emergent capability of just scaling up the multimodality of the model."
- Initially, models from OpenAI and Anthropic refused to issue motion commands to the physical car, responding that they could only interpret road images, but the engineers coaxed them into driving with careful prompting after first trying Grok; one engineer quipped "Maybe AGI is here after all" as Astra reached the take-out window.
- Elorian AI, a startup founded by former Google DeepMind researcher Andrew Dai, partnered with Scale AI on a new benchmark called Humanity's Sixth Sense that measures models' intuitive understanding of physical scenes beyond raw perception.
- The engineers acknowledge that putting a general-purpose language model in control of a "fast-moving, two-ton hunk of steel" is high-stakes, even though nothing alarming happened during the lunchtime run.
Why it matters: DrivingBench's results suggest physical driving ability is emerging as an unintended byproduct of multimodal training, not deliberate engineering — meaning AI labs chasing 3D reasoning could inadvertently produce systems that control real-world machinery. For self-driving incumbents like Tesla and Waymo, whose specialized models were thought to hold a defensible moat, this hints that general-purpose AI may close the gap from an unexpected direction.
Ask SkimNews
