We still don’t know how people are really using AI
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Anka Reuel, a Stanford PhD candidate, co-leads the AI Observatory, a public platform that aggregated 24,521 conversations across 85,633 conversational turns from 5,000 consenting users interacting with 52 models — including Claude, Gemini, ChatGPT, and Grok — between 2023 and 2025, drawing on seven existing datasets.
- AI Observatory researchers found that 48% of conversations would be filtered out using Anthropic's Economic Index methods; the filtered conversations showed far higher rates of health and relationships topics (44.2% vs 31.2%), adult or illicit content (7.9% vs 2.1%), harassment and hate (27.5% vs 5.66%), and sexual content (16.7% vs 2.4%).
- OpenAI's 2025 report on ChatGPT found that only 30% of consumer use was work-related, reinforcing the Observatory's claim that company-curated reports understate non-work uses.
- AI usage patterns differed sharply by model: Grok and Gemini led for information retrieval, with Grok a hotspot for news, politics, and misinformation; Anthropic drew coding use; Gemini drew social and roleplay use; ChatGPT drew homework help; and ChatGPT conversations grew longer and more iterative from GPT-3.5 to GPT-4o.
- WildChat conversations grew longer with more small talk over time — suggesting rising AI companionship — while AI self-disclosure decreased and sensitive-use exchanges dropped, which researchers said may indicate platforms deploying more effective safeguards.
Why it matters: Policymakers and researchers currently make consequential decisions about AI's risks based largely on company-curated usage reports — but the AI Observatory found 48% of conversations are filtered out by Anthropic's work-focused methods, including disproportionate shares of harassment (27.5%), sexual content (16.7%), and illicit topics (7.9%). Regulators may therefore be systematically blind to AI's most sensitive real-world applications, and independent researchers now have a public dataset of 24,521 conversations to fill that gap.
Ask SkimNews
