"Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- GPT-5.6 Sol won the drawing arena across 28 total drawings (Mona Lisa, Starry Night, and five text prompts), with the authors calling it the "runaway leader" whose Starry Night and rose were their favorites of the entire batch.
- Claude Fable 5 cost roughly $160 for its seven drawings — about 20x more than GPT-5.6 Sol ($7.74), Grok 4.5 ($9.21), or Gemini 3.6 Flash ($12.87) — yet the authors ranked its output second on quality, calling it "the least appealing trade-off of the three."
- Gemini 3.6 Flash posted the highest structural-similarity (SSIM) scores on both target images, peaking at 0.449 on the Mona Lisa, but also swung the widest between peak and final — finishing that drawing at 0.337.
- Across all eight target runs, the models over-edited past their own best work: every final drawing scored below the model's mid-run peak, and Gemini's ~23 self-reviews per drawing — the most of any model — did not translate into a better finish.
- Grok 4.5 was characterized as "basically garbage," with 65% of its 1,349 tool calls spent on color/brush/pressure settings rather than actual drawing; the authors also noted that open-weight models they tested were "not even usable, several just returned a blank canvas."
- All four models used the same 10-tool colored-pencil harness (draw, smudge, erase, view_canvas, etc.) on a blank canvas, and the full setup is open-sourced at github.com/hershalb/canvas-arena for anyone to run against a target image or text prompt.
Why it matters: Practitioners running long-horizon visual or tool-use agents now have concrete cost-versus-quality data: Claude Fable 5 ran roughly 20x more expensive than the other three models for output the authors ranked second, and every model wasted compute editing past its own peak SSIM — a pattern that directly affects per-step billing for agentic workflows.
