Letting the blind navigate their environment using sound.
As VR and mixed reality headsets continue to make their way into the hands of enthusiasts around the world, we are continually impressed with what developers dream up to help improve the lives of others. From exploring the use of VR for medical training to discovering new ways immersive technology can impact prosthetic patients, exciting progress is being made on a daily basis.
Now one Portland-based HoloLens developer, Javier Davalos, is showing us what the mixed reality hardware can do to help the visually impaired.
In a demonstration video shared on YouTube, Davalos walks us through his latest app for the Microsoft HoloLens mixed reality headset that gives the user audio feedback to assist in navigating their environment.
In the case of this demo, the app spatially maps the room for Davalos to look in any direction and determine the distance of objects through sound. The idea is like a bat’s biological sonar where sounds bounce back like an echo to allow them to detect obstacles in their path or locate food. But for Davalos’ HoloLens app, sounds get louder or softer depending on how far away you are from an object.
In the first minute of the demonstration, Davalos shows a monoscopic version of the app where you have the ability to hear different sounds depending on what is in front of you. The floor makes a sound different from the floor. And other objects in the room also have a unique sound as well. In a way, it could be possible for users to identify objects over time just through the specific sound feedback.
It’s when Davalos switches the app to stereo mode, or “two eyes” according to him, that things get even more exciting. Now instead of audio feedback from a single point in front of you, two points project sound into your left and right ear, allowing for added object awareness like determining edges or maybe where a wall ends. Moving your head left to right over a wall edge gives you a more accurate understanding of where the wall ends and a hallway begins.
In addition to stereo mode, Davalos shows how he built in simple voice commands into the app that can let you hear distance measurements and which direction North lies. He also used the app to count the number of stair steps in front of him.
Davalos has been busy at work developing apps for the HoloLens, everything from home interior design to an actual gardening tool, but he’s asking for your help on this visual assistant app. He is currently looking for someone with visual impairment to test the application and see how the app can be improved upon. In the mean time, we can’t wait to see where the project goes.
