AI Stories Outrated Human Ones; Readers Couldn't Tell
SkimNews Take
A "human-written" label still outpulls AI prose even when readers rate the AI version as more absorbing — exposing a residual authenticity premium that label-blind evaluation fails to capture.
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- The study, published in the journal Judgment and Decision Making, had 1,682 adults read one of six short stories — three human-written, three by ChatGPT — with each AI piece matched to a human counterpart on theme.
- Participants who read AI-generated stories rated them as more absorbing and of higher quality than those who read human-written stories, according to the researchers.
- Stories labeled as human-written received higher scores regardless of actual authorship, exposing a bias toward human labels — a tension the source surfaces but the dominant "AI wins" framing flattens.
- In two follow-up experiments with 905 adults asked to identify which of two same-themed stories was human-written, 40% guessed correctly in one round (worse than chance) and 52% in the other, showing no reliable ability to distinguish AI from human prose.
- Dr. Deena Skolnick Weisberg of Villanova University said AI can now produce stories "at least as good as – if not better than – human-written stories" and that views of AI's capabilities "should update."
- Luke Kennard, professor of film and creative writing at the University of Birmingham, pushed back, calling AI not a tool but an "existential threat" built on "a massive stolen database of actual writing" and housed in "destructive datacentres."
- Attitudes toward AI also shaped the ratings: participants with more positive views of AI scored stories they were told were ChatGPT-written higher than those with negative AI attitudes.
Why it matters: The 40% accuracy rate — worse than chance — in telling human from AI writing undercuts the assumption that readers can spot machine prose, with direct stakes for literary markets, classrooms, and publishing workflows already receiving AI submissions. Kennard's counterpoint adds that the gains rest on uncompensated human training data and energy-intensive infrastructure, a cost the ratings alone do not capture.
Ask SkimNews
