Chat LLMs replicate psychic cold-reading con, essay argues — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Chat-based LLMs replicate the mechanism of a psychic's con (cold reading), using validation statements that sound personally specific but are statistically generic — the same Forer/Barnum-style tactics used by mentalists.
- The author identifies only two possible explanations for perceived LLM intelligence: either the tech industry accidentally invented an entirely new kind of mind, or the 'intelligence illusion' lives in the user — the author firmly supports the latter in their book 'The Intelligence Illusion.'
- LLMs are mathematical models of language tokens that produce mathematically plausible responses, and the author notes that every AI researcher and vendor to date has repeatedly emphasized these models do not think or reason.
- Terence Eden's blog post on the prevalence of Forer statements in chatbot replies triggered the comparison, reinforced by AI believers' language — 'This is real. It's a bit worrying, but it's real' — which mirrors the words of victims of mentalist scams.
- The psychic con operates through six documented stages: audience self-selection, scene-setting, demographic narrowing, mark testing, a subjective validation loop, and the mark concluding the psychic has uncanny powers.
- Subjective validation — the cognitive bias that makes people judge generic statements as personally accurate if they carry any personal meaning — is identified as the core shared mechanism behind both cold reading and the chatbot 'intelligence illusion.'
- The author now believes there is even less intelligence and reasoning in LLMs than previously thought, and characterizes many proposed use cases as 'borderline fraudulent pseudoscience.'
Why it matters: The author — who has written a book on generative AI risks titled 'The Intelligence Illusion' — now calls many proposed LLM use cases 'borderline fraudulent pseudoscience,' directly contradicting the sales narrative of AI vendors who insist their models possess reasoning abilities.
Ask SkimNews
