Sign-IO: Roy Allela's Gloves That Turn Sign Language Into Speech

Facebook ↗WhatsApp ↗
Roy Allela’s Sign-IO Gloves Explored Turning Hand Movements Into Speech

Sign-IO is a pair of smart gloves developed by Kenyan engineer Roy Allela that turns hand movements into spoken words. Sensors along each finger measure how far it bends, a phone app linked by Bluetooth works out which sign is being made, and the phone speaks it aloud. Allela built it after struggling to understand his deaf niece, tested it at a special-needs school in rural Kenya, and in 2019 the project was shortlisted for the Royal Academy of Engineering's Africa Prize for Engineering Innovation.

It is an inspiring piece of engineering, and it also raises good questions about what technology can and cannot do for sign language users. Both are covered below.

Who Roy Allela is

Allela was in his mid-twenties when the gloves made international news in 2019. Reports at the time described him as a software engineer working with Intel and a tutor in data science at the University of Oxford. The University of Bristol later featured him on its "Engineering Includes Me" wall, which celebrates people from under-represented backgrounds in science and engineering.

The motivation was personal. His young niece is deaf. She could follow what he said by reading his lips, but he and most of the family could not understand her sign language. Communication ran only one way, and Allela wanted to fix that.

How the Sign-IO gloves work

Roy Allela’s Sign-IO Gloves Explored Turning Hand Movements Into Speech
Roy Allela’s Sign-IO Gloves Explored Turning Hand Movements Into Speech
Photos: African Curators

Each glove carries flex sensors stitched along the fingers. When a finger bends, the sensor's electrical resistance changes, and the degree of bend is measured. The pattern from all five fingers is sent wirelessly to a smartphone app, which matches it to a letter or sign and plays the result as speech in near real time.

Users can adjust how the voice sounds: its language, gender and pitch, plus the speaking speed. Allela also made the gloves themselves customisable, with children able to pick designs such as superheroes or princesses. He has said that making the device look fun helps reduce the stigma of wearing assistive technology, because other children become curious instead of dismissive.

The team reported 93 percent accuracy. That figure is promising, but it applies to the set of signs and test conditions the team used. It does not mean the gloves can understand 93 percent of free conversation in any sign language.

Testing in Kenya and the awards

Roy Allela’s Sign-IO Gloves Explored Turning Hand Movements Into Speech
Photo: Roy Allela

Allela introduced the gloves at a special-needs school in Migori County, in rural south-western Kenya. Feedback from the pupils showed that the main weakness was speed: translating sign to voice took too long to feel like natural conversation. That became the next engineering priority.

The project won the Hardware Trailblazer award at the American Society of Mechanical Engineers' Innovation Showcase (ISHOW), and the Royal Academy of Engineering included Sign-IO in its 2019 Africa Prize shortlist of 16 innovators, describing it as a glove that "tracks and translates sign language movements into speech in real time." Allela said he would put prize money into improving the app's predictions, and he hoped to place gloves in every special-needs school in Kenya.

One correction to figures that circulated with the story: claims that "over 30 million people" have speech impairments were not sourced. The World Health Organization estimates that about 430 million people worldwide have disabling hearing loss, with nearly 80 percent living in low- and middle-income countries. Not all of them use sign language, and many people who sign are not deaf.

What sign language gloves cannot do

Sign-IO was not the first glove of its kind, and the whole category has drawn careful criticism, often from Deaf people and sign language researchers. A widely read 2017 article in The Atlantic by Michael Erard set out the main concerns:

  • Sign languages are not just hand shapes. Grammar is carried by facial expressions, head and body movement, the position of the hands in space and the direction of movement. A glove that reads finger bends misses most of this.
  • Fingerspelling is not conversation. Spelling words letter by letter is slow. Fluent signing uses whole signs, and every country has its own sign language with its own vocabulary and grammar.
  • One-way design. A glove helps hearing people understand the signer but puts the burden of wearing equipment on the Deaf person. It does not help the signer understand the hearing person.

Researchers elsewhere, including a UCLA team, have tried adding facial sensors to capture more of the language. None of this makes Allela's work less impressive; it simply defines what a glove can realistically be: a helpful tool for specific situations, not a replacement for interpreters or for hearing people learning to sign.

Kenyan Sign Language and the bigger picture

Roy Allela’s Sign-IO Gloves Explored Turning Hand Movements Into Speech
Photo: African Curators

Kenya has its own sign language, Kenyan Sign Language (KSL), used by a large share of the country's estimated several hundred thousand deaf people. Its manual alphabet draws mainly on the American one, which matters for any device built around letters. Kenya's 2010 constitution names KSL among the languages the state should promote, but deaf children, especially in rural areas, often still lack teachers and family members who sign fluently.

That is why projects like Sign-IO get attention. In places where interpreters are rare, even a partial bridge between a signing child and a hearing family has real value. The most useful thing hearing readers can take away is that the best "technology" for that bridge remains learning some sign language, starting with the local one.

Why the story still matters

Allela started with a single relationship, between himself and his niece, and built a working prototype around it with low-cost sensors and a phone. He tested it with real users, listened when they said it was too slow, and was honest that it needed more work. That is good engineering practice anywhere. The open questions raised by Deaf critics are part of the same story: the people who use a tool should decide what success looks like.

Sources: Royal Academy of Engineering: 2019 Africa Prize shortlist; City Press: Smart glove reads sign language; University of Bristol: Engineering Includes Me, Roy Allela; WHO: Deafness and hearing loss; Wikipedia: Sign language glove; Wikipedia: Kenyan Sign Language; photos by Roy Allela and African Curators.

COMMENTS