Brain implant decodes speech and gestures at once — SkimNews
Get the Health newsletter
Daily health & science — research, biotech, public health, the studies worth knowing. Free.
- Samantha Brosler and colleagues at UC San Francisco tested an iPhone-sized brain implant in the sensorimotor cortex, decoding simultaneous speech and gestures from two people with paralysis at 66-88% accuracy — far above the ~9% chance baseline.
- Bravo-1r, paralyzed by a stroke, had gestures predicted at 88% accuracy and speech at 84%; Bravo-6, who has ALS, hit 66% on gestures and 70% on speech, a notable gap between conditions.
- Both participants had lost nearly all limb movement and could only produce unintelligible sounds, yet attempted 10 phrases like "hello" and "nice to meet you" alongside 10 gestures including waving, clapping, and head-shaking.
- Customized machine learning models trained on each participant's brain recordings were used to drive an avatar's movements while flashing speech predictions on screen.
- The team chose avatars over direct speech and movement, calling it a "relatively simple way to prove" the dual-channel decode worked — a proof of concept rather than a deployable product.
- Henri Lorach of the University of Lausanne called the work "a very nice piece of work, with robust results" but said the limited 10-phrase, 10-gesture vocabulary needs higher accuracy "for daily use without frustration."
- Future models could move a person's own body (if their joints allow) and produce a synthetic voice rather than relying on an avatar, Brosler said.
Why it matters: Two patients with paralysis from different causes — stroke and ALS — were the first to have simultaneous speech and gesture decoded from the same brain region, hitting 66-88% accuracy against a ~9% chance baseline. But the 10-phrase, 10-gesture vocabulary and the 22-point accuracy gap between the two patients underscore how far the field still is from frustration-free daily communication.
Ask SkimNews
