Microsoft teases practical MR worker tools for Hololens at 2018 Build Developer Conference.
Call me strange, but of all the mixed reality demos I tried at this year’s Microsoft Build Developer Conference in Seattle, Washington, the one I actually found the most fun had me repairing an electric panel. To truly comprehend the deep sense of satisfaction I experienced as the pretend machinery setup at the Seattle Sheraton hummed to life, you first have to understand how utterly useless I am at any form of DIY. So even though I’d been guided through the procedure step-by-step, I felt a real sense of accomplishment at the end, even if it was all make-believe.
I was trying out the Microsoft Remote Assist feature, a HoloLens app (although the functionality also works on PCs. Phones and tablets) designed to help Firstline workers collaborate remotely with hands-free video calling, image sharing, and a full complement of mixed reality annotations. This type of use-case scenario is not new in itself, as HoloLens works with Skype out of the box which of course enables you to video call someone, have them “see through your eyes” and offer advice on what you should do. In fact, German manufacturing giant Thyssenkrupp has been using this exact functionality to improve the efficiency of their elevator repair crews for well over a year now.
What is different about this new method however, apart from being an overall more polished experience capable of integrating more collaborative features via Microsoft Teams, is that it also plugs into the company’s strategic focus on AI. The idea is the more enterprise customers that use the service, the more the system will be able to learn from the data, in-turn providing better optimization for the experience (for example: recommending the best expert to contact depending on the type of problem encountered and their availability in real-time, taking into account what time zone each user is in, etc.)
During the demo, the remote expert I had on video call not only direct me on what to do based on what I was seeing via the HoloLens, but was also able to search for a relevant schematic map, pin it to the wall, and then draw holographic circles around the parts of the electric circuits that needed my attention. It was an extremely smooth experience made all the better by that rush of confidence you get from successfully completing a task you once thought out of your skill range.
The other advantage for enterprise customers looking to use this particular service is Microsofts promise to provide advanced identity and security measures intended for more discrete settings in which businesses can privately share plans and other reference materials relating to industrial processes.
A similar demo I tried in Seattle came from 3D app developer Taqtile. This particular app generates step-by-step instructions supported by images, videos, text and holograms, all integrated into real-world physical actions. The demonstration differentiated itself from the previous experience by providing me with zero assistance from a real-world instructor. Despite this lack of guidance, and my reputation for being the kind of person who follows the LEGO instructions to the letter and still ends up with a wonky masterpiece every time, I still managed to navigate the process all the way to the end and lo and behold, the light switched on! It’s all about those simple pleasures.
Taqtile’s interface makes it easy for anybody to create such a list of instructions, which can then be used in a variety of scenarios from training and simulation, to verification and compliance.
In the United States Air Force for example, jet engine mechanics are not permitted to utilize operational airplanes for training. Some military bases have training engines available, but these don’t necessarily match the aircrafts they would be called upon to repair. Many mechanics are also restricted to learning from manuals and other supporting documentation, with their only hands-on experience happening as they’re actually performing those tasks on aircraft that are in service.
This step-by-step training, however, allows those same students to practice alongside a physical training engine or a virtual 3D holographic one. This means they are physically performing and rehearsing relevant actions, building muscle memory as well as improving their confidence and knowledge retention.
Those same materials can also be used by fully trained technicians as walk through a job site and tick each step off the list. A mechanic in the field can select an appropriate job from the work order system accompanied by step-by-step instructions (overlaid on the physical equipment) on how to complete the task, and then check off the job after it’s been completed. These instructions can include media such as video, audio, and images, even links to external information sources as well as 3D models of the equipment’s “digital twin” and sub-parts.
So rather than having to rifle through a binder to find the appropriate Material Safety Data Sheet, a mechanic can access it directly from the relevant step in the augmented tutorial. The inspector function allows a supervisor or compliance technician to review the mechanic’s work. The inspector can bring up the job completed by the technician, and can walk through step-by-step to ensure that everything was executed properly.
Mixed reality applications such as these can really help address a cross-industry challenge – a so-called plateau in productivity growth which results from a decline in the number of skilled workers as well as an increase in demand for maintenance and inspection expertise.
So while I enjoy a good VR rollercoaster or a spell of realistic zombie-blasting as much as the next gal, I do believe these are the sort of practical, functional immersive experiences that will sustain the growth of immersive technology as consumers begin to wrap their heads around how this technology fits into their day-to-day lives. These applications give companies an immediate and often dramatic return on their investment, and – if my own experience is anything to go on – they can also be rather fun.
Image Credit: Microsoft Build
