Skip to main content

Amazing new glove can translate sign language into spoken words in real time

Wearable Sign-to-Speech Translation

Researchers at the University of California, Los Angeles, have developed a wearable device, resembling something approaching the Nintendo Power Glove, that’s able to translate American Sign Language into speech in real time using a smartphone app. While it’s still in the prototype phase, it could one day help those who rely on sign language to communicate more easily with non-signers, along with assisting novices who are learning sign language.

“Analog triboelectrification and electrostatic induction-based signals generated by sign language components — including hand configurations and motions, and facial expressions — are converted to the digital domain by the wearable sign-to-speech translation system to implement sign-to-speech translation,” Jun Chen, assistant professor of bioengineering at the UCLA Samueli School of Engineering, told Digital Trends. “Our system offers good mechanical and chemical durability, high sensitivity, quick response times, and excellent stretchability.”

The gloves contain thin, stretchable sensors made of electrically conductive yarn which run along the length of all five fingers. They communicate the finger movements of the wearer to a small, coin-sized circuit board that’s worn on the wrist, which in turn transmits the data to a connected smartphone. Because American Sign Language relies on facial expressions in addition to hand movements, the system also involves sensors adhered to users’ eyebrows and the sides of their mouths. Built around machine learning algorithms, the wearable is currently able to recognize 660 signs, including every letter of the alphabet and numbers zero through nine.

ASL reading system 1
University of California, Los Angeles

Chen said that previous sign language translation devices have been based on a wide range of techniques, including electromyography, the piezoresistive effect, ionic conduction, the capacitive effect, and photography and image processing. But the inherent complexity of these tools, in addition to how cumbersome they are, has made them little more than proof-of-concept lab experiments.

“For example, vision-based sign language translation systems have high requirements for optimal lighting,” Chen said. “If the available lighting is poor, this compromises the visual quality of signing motion captured by the camera and consequently affects the recognition results. Alternatively, sign language translation systems based on surface electromyography have strict requirements for the position of the worn sensors, which can impact translation accuracy and reliability.”

The hope is that this wearable sign-to-speech translation system could be more realistically used in real-world settings. In addition to not being affected by external variables like light, the UCLA sign language wearable could be produced inexpensively. “We are still working to polish the system,” Chen said. “It may take three to five years to get it commercialized.”

A paper describing the work was recently published in the journal Nature Electronics.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Here’s a shocking reminder of just how far ahead Intel is in race with AMD
Intel Core i5-14600K processor inside its socket.

It's no secret that between Intel and AMD, it's Intel that holds the majority of the market -- but a recent market analysis report really highlights just how much Intel beats both AMD and Apple. While both Intel and AMD make some of the best CPUs, and Apple has its own excellent proprietary silicon, Intel's CPU shipments trump them both. The report tells us that Intel controls a whopping 78% of the PC processor market.

The information comes from Canalys, a PC industry research firm. It's worth noting that it excludes tablets, which would have bumped up Apple's numbers a bit, as well as Qualcomm's. But zooming in on the PC CPU market alone, including desktops and laptops, tells a different story. The research shows a 78% market share for Intel and 13% for AMD, while Apple accounts for about 8% or slightly more. The rest belongs to MediaTek, Qualcomm, and ARM.

Read more
Beware! The latest Windows 11 update might crash your PC
A laptop running Windows 11.

Microsoft releases routine updates for Windows every month, and while the intent is to fix issues and occasionally add new features, the latest one is doing more harm than good. Some Windows 11 users have taken to social media to report that the latest KB5035853 update could be crashing their PCs, with the feared Blue Screen of Death (BSoD) appearing on startup.

Although Microsoft hasn't appeared to document any known issues with this update for Windows 11 22H2 and Windows 11 23H3, there's a Reddit thread on the matter. Frustrated Windows 11 users mention the problems they are experiencing after installing the update. One mentions that Explorer.exe shows an error during shutdown and another reports that the ribbon of the File Explorer is slow to load. Audio-related issues and PCs feeling laggy when gaming are among some of the other problems being reported.

Read more
We finally know the price of Asus’ most powerful gaming NUC
The Asus ROG NUC on a desk surrounded by three monitors.

The first Asus ROG NUC (Next Unit of Computing) model is just around the corner. The small form factor PC is now up for pre-order at a German retailer, and although it's powerful enough to rival some of the best laptops, it costs more than many comparable models -- and you'll still have to pay extra for a monitor.

Asus' first take on Intel's portable PC contains a lot of compute power in a small chassis. Although there are a few configurations of the PC, the one that was spotted up for sale ahead of time comes with Intel's latest Meteor Lake-H CPU, the Core Ultra 9 185H, which sports 16 cores and 22 threads and can be boosted to run at up to 5.1GHz, all with a thermal design power (TDP) of 45 watts. However, Asus allows overclocking, meaning that the CPU can run at up to 65 watts instead.

Read more