Study: People Prefer AI Stories and Can't Tell Them Apart
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Sydney Sears and Deena Weisberg at Villanova University surveyed 1,682 US-representative adults and found AI-generated stories scored 1.54 on quality vs. 0.97 for human-written stories (on a -3 to 3 scale), and 1.42 vs. 1.0 on 'absorbing' qualities.
- The study's 2×2 design told participants their story was either human- or AI-written regardless of true origin — and every story was rated slightly higher when participants believed it was human-written.
- Two follow-up discrimination tests found only 39% and 52% of participants correctly identified which story was AI-generated, which the researchers called no better than chance; the improvement between tests may reflect society adapting to AI output.
- Claire Hardaker at Lancaster University called it an 'uncomfortable' finding that humans aren't better than robots at short stories, and noted that people react most angrily to AI-generated music when deceived.
- Rodney Jones at University of Reading suggested the result may reflect that literary journal writing is 'hard work' to process, while ChatGPT defaults to simpler, easier-to-read 'vanilla' content.
- Weisberg plans future experiments on different writing genres and argued that 'AI creativity is just different from human creativity,' predicting AI will produce work with artistic value 'measured on a different scale.'
- The surrounding backlash includes a Commonwealth prize-winning Granta story accused of being AI-generated (investigation inconclusive) and artists protesting an AI art auction at Christie's as 'mass theft.'
Why it matters: For writers, publishers, and literary gatekeepers, the finding that 1,682 readers preferred default ChatGPT stories over curated literary journal pieces — and couldn't identify them at better-than-chance rates — undermines the assumption that human literary writing has a recognizable quality edge, with direct implications for how AI-generated literature is judged, valued, and policed.
