Show HN: Gemini 4 Argon plays StarCraft — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- StarSkirmish Bench v0.2 tests LLMs by having each one write a Protoss StarCraft: Brood War bot in C++ within one hour, with v0.2 switching to each lab's own 1st party CLI harness at maximum reasoning effort
- GPT-6 Astra tops the benchmark, with GPT-6.1 Sol second; Gemini 4 Argon, Claude Opus 5.5 and Claude Fable 5.1 are functionally tied behind them, all scoring between 37 and 39
- GPT-6.1 Sol sits alone in the best value quadrant, scoring second at roughly a tenth of GPT-6 Astra's per-run API cost
- Human-written bots still dominate the field: against Stardust, PurpleWave and BananaBrain, GPT-6 Astra wins only 7% of games
- Practice against strong bots is the strongest predictor of final score; GPT-6 Astra, GPT-6.1 Sol and Gemini 4 Argon started running practice games against top-tier opponents early in the hour, while still losing to them
- Gemini 4 Argon shows the widest run-to-run variance, with best and worst runs 18 points apart, versus 9 for GPT-6 Astra and 8 for Claude Opus 5.5
- StarSkirmish scores now barely correlate with the Artificial Analysis Intelligence Index (Spearman ρ 0.26) and have drifted away from Terminal-Bench 4.0, while still tracking Vals IOI and WeirdML v3
Why it matters: For AI labs marketing long-horizon agentic coding, the headline gap is humbling: GPT-6 Astra wins just 7% vs. top human-written StarCraft bots. StarSkirmish scores now barely correlate with the Artificial Analysis Intelligence Index (ρ 0.26), suggesting the most-cited evals may be approaching saturation or tuning targets while tougher long-horizon coding tasks still surface real differentiation. Buyers weighing cost-to-performance should notice GPT-6.1 Sol nearly matches the leader at roughly one-tenth the per-run price.
Ask SkimNews
