Brain Implant Turns Imagined Handwriting Into Text: The 2021 Study

Facebook ↗WhatsApp ↗

First published November 11, 2021.

In 2021, a team in the BrainGate research consortium showed that a man paralysed from the neck down could produce text on a screen simply by trying to write letters by hand. Tiny sensors in the movement area of his brain picked up the activity, and software turned each attempted pen stroke into a letter at about 90 characters per minute, with roughly 94 per cent of characters correct before any autocorrection. The study, led by Frank Willett at Stanford and published in Nature, was a proof of concept with a single volunteer. It did not read his private thoughts, and it was not a product anyone could buy.

A Brain-Computer Interface Decoded Attempted Handwriting in a 2021 Study

Who took part and what he did

The volunteer, known in the research as T5, was 65 at the time. A spinal cord injury in 2007 had left him with almost no movement below the neck. He was enrolled in BrainGate2, a long-running clinical trial of implanted brain-computer interfaces, often shortened to BCIs.

Neurosurgeon Jaimie Henderson placed two small electrode arrays, each roughly the size of a baby aspirin, in the part of the motor cortex that controls the right arm and hand. During sessions T5 was shown a prompt, imagined holding a pen above a pad of paper and tried to write each letter, as if his hand still worked. His arm did not move. The arrays recorded the electrical activity of many individual neurons while he made the attempt.

How the system turned brain signals into letters

The key insight was that handwriting is a rich, varied movement. Each letter involves its own pattern of curves, loops and changes of speed, so the neural activity behind an "a" looks quite different from the activity behind a "b". A recurrent neural network, a type of machine-learning model that is good at sequences, was trained to recognise those patterns and output the letter T5 was trying to write. The character set covered the 26 lower-case letters and a few punctuation marks.

Willett described this as a surprise worth noting. In his words, the brain "retains its ability to prescribe fine movements a full decade after the body has lost its ability to execute those movements." Complex, curved motions proved easier for the algorithm to tell apart than the simple straight-line cursor movements earlier BCIs relied on.

On top of the letter decoder, the team used a language model, the same idea behind autocorrect on a phone, to fix likely mistakes. That second step matters when reading the headline numbers.

What the numbers mean

Headlines at the time spoke of "94 per cent accuracy" and "99 per cent with autocorrect". Both figures come from the study, but they measure different things:

  • Speed: about 90 characters, or roughly 18 words, per minute when copying sentences. The previous record for a BCI typing system was about 40 characters per minute, using a point-and-click virtual keyboard.
  • Raw accuracy: around 94 per cent of characters correct in real time, with no language model help.
  • Corrected accuracy: above 99 per cent when a language model was applied afterwards, in offline analysis rather than live use.
  • Comparison: people of T5's age typing on a smartphone average about 115 characters per minute, so his rate was in the same range.

T5 also answered open questions in his own words, at a somewhat lower speed. These results come from one person over a limited number of sessions, so they show what is possible rather than what every user would achieve.

Myths: what this technology does not do

It does not read your thoughts. Early headlines said the implant "transcribed thoughts". It did not. The system decoded deliberate attempts to make a hand movement, in one area of the brain, after training on that person's signals. Thinking a word silently, without trying to write it, would not produce text.

It is not a headset. The arrays sit inside the skull and require brain surgery. Non-invasive caps that read signals through the scalp are far less precise.

It was not ready for hospitals. Some coverage suggested the device would soon be widely used. The researchers themselves framed it as a step forward, with more work needed: testing it with more people, adding capital letters and symbols, improving editing tools and making the hardware wireless and durable for daily life at home.

BrainGate in context

BrainGate grew out of work at Brown University and the company Cyberkinetics in the early 2000s. In its first human trial, published in Nature in 2006, Matt Nagle, a young man paralysed by a spinal cord injury, used an implanted array to move a computer cursor, open email and change television channels. In 2012 the consortium reported that two people paralysed by brainstem strokes could control robotic arms; one of them, Cathy Hutchinson, used a robotic arm to drink from a bottle on her own for the first time in about 15 years.

The trial is coordinated through Massachusetts General Hospital, Brown, Stanford, Case Western Reserve University and the US Department of Veterans Affairs. Leigh Hochberg, who directs it, has described the goal as restoring rapid, intuitive communication to people who have lost the ability to move or speak. In April 2021 the team also reported the first wireless transmission of such brain signals from people with paralysis, with performance close to wired systems.

Why it matters

For people with advanced ALS, brainstem stroke or high spinal cord injuries, communication can shrink to eye blinks or slow letter boards. A system that lets someone write at near-smartphone speed could restore privacy, independence and the ability to hold a real conversation. The handwriting study also changed how researchers think about decoding: rather than relying only on simple movements, they can tap into complex skills the brain still remembers.

Several companies and academic groups have since pushed this field forward, including work on decoding attempted speech. If you or someone you care for lives with paralysis and is curious about these trials, the best first step is to talk to a doctor, who can explain whether any study is suitable and what the surgery would involve.

Sources: Willett et al., "High-performance brain-to-text communication via handwriting", Nature, 2021; Brown University news, 12 May 2021; ScienceAlert, November 2021; Wikipedia: BrainGate.

COMMENTS