AI Builds Shadow Medical System Outside Hospitals
Get the Health newsletter
Daily health & science — research, biotech, public health, the studies worth knowing. Free.
- ChatGPT receives health questions from more than 40 million Americans every day, mostly outside clinic hours, and most users never consult a doctor afterward, according to the authors.
- A published study found that medical disclaimers in chatbot health answers have largely disappeared, and today's leading models now ask follow-up questions and attempt diagnoses rather than redirect users to clinicians.
- Doctronic has run 24 million AI consultations and now writes AI-generated prescription refills in Utah; Function Health, valued at $2.5 billion in November, lets members order 160 lab tests per year and authorize ChatGPT to read the results.
- Oura sells a 50-biomarker blood panel through Quest Diagnostics for $99, while Ro and Hims prescribe weight-loss or anxiety medications after asynchronous intakes with no in-person exam.
- A JAMA Network Open study testing 21 frontier models found they named the correct diagnosis more than 90% of the time given a complete case, but failed to produce a comprehensive differential more than 80% of the time using only what a clinician would gather at the start of a visit.
- Anthropic, OpenAI, and Google have all launched mechanistic interpretability research programs, but the field remains nascent, especially for clinical concepts, and engineers cannot fully explain how their own models arrive at answers.
- Healthcare represents nearly one-fifth of the U.S. economy, and the authors argue many new health AI products are "little more than a polished user interface wrapped around someone else's model," borrowing medicine's authority while avoiding its accountability.
Why it matters: AI's strongest results come from controlled studies with complete clinical data, yet consumer health tools are deployed for the hardest real-world task: initial triage from scattered, unfiltered patient symptoms. With an 80%+ differential diagnosis failure rate from limited input and no accountability framework, these products risk guiding patients away from clinicians precisely when they need them most.
Ask SkimNews
