With Kokomo VR meeting software, Canon takes a step away from its hardware roots

Canon has a long and deep history of being a hardware manufacturer. Most consumers know it best as a camera manufacturer, but the company has a long, deep and illustrious history in medical, office equipment, and other imaging applications.

During the pandemic, a lot of its business shifted. People stopped going to offices. Sporting events were shut down. And while the medical industry was booming, Canon as a company needed to rethink its mission and vision: What does an imaging company do in a world where people have a desire to connect, but are unable to leave their homes while a deadly virus rages around the world?

At CES 2023, the company showed off its vision for the future — a vision that seems a lot less hardware-y than you would expect from the 85-year-old company that has traditionally made all of its money from making things with buttons.

A rag-tag bunch of Canon veterans took on the challenge and created Kokomo, a VR meeting software package that, in essence, makes real-time 3D video calling a reality.

Users don a VR headset and point a smartphone at themselves. The software scans your face, and creates a photo-real 3D avatar of you and the person you are calling. It uses the motion sensors in the headset and the camera to capture your avatar, moving you into a photo-realistic space, and boom, you are virtually present with a colleague, family member or friend. The technology to scan your face is similar to the tech used by iOS’s face ID, doing a brief pre-capture process. From there, your face’s shape and texture can be shown off inside the video calls.

The most interesting thing to note about the above paragraph is the lack of Canon products. Traditionally, Canon’s software solutions have focused on enhancing and facilitating the use of its hardware products. The company makes neither smartphones nor VR headsets, however, so this move represents a major departure from its roots.

TechCrunch sat down with the team that led the development of Kokomo to figure out how it came about, and where Canon is going as it re-imagines its own future.

“This is representing a very exciting new innovation for Canon – but also a very new business direction for Canon, as well,” said Jon Lorentz, one of the co-creators of the Kokomo solution. “As you know, traditionally, Canon is very much tied to our hardware products. When we announced AMLOS at CES last year, it was about innovating work from home. Our task [with Kokomo] is to innovate life at home, and that is where this project came from. When we started, we were in the thick of COVID, and there were not a lot of ways for people to get connected. The underlying premise for what we created was to be a solution to that. Kokomo is a way to enable people to be there when they couldn’t.”

The team’s goal was to create a solution that takes the experience beyond a phone call, a FaceTime call or a Zoom call – to feel present rather than to just look at each other on a screen. A worthy pursuit in a world where travel is limited and screen fatigue is real. But how is Canon’s solution for bringing people into a virtual world going to accomplish that?

“We support most of the popular consumer VR headsets in the market to enable people to engage in immersive calls, as we are calling them. In these calls, people can engage. They are dynamic, in living, breathing environments. You can download a companion app on a mobile phone, which lets the person you talk to see you from head to toe,” explains Lorentz. “No more legless avatars. No more wondering what someone is actually gesturing. And you can actually see the other person. You can be in the call, rather than on the call.”

Below is an in-depth interview with Kokomo co-creators Jon Lorentz, Ray Konno and Jason Williams. The interview has been edited for clarity and length.

TechCrunch: Why is Canon excited in software? Isn’t that a step away from its hardware roots?

Jon Lorentz (JL): At our core, Canon is an imaging company, and that’s really our specialty. Kokomo is applying that specialty to the software rather than starting with our hardware first. We see that the ability to step into a call is really stepping into an imaging sensor. It’s about taking that image sensor data, and then applying it to someone else’s visual field.

Obviously, there are a lot of details behind that, but our core is imaging excellence. As you bring in mesh reality and virtual reality, you need to have a certain level of meshing: it really needs to match up. Otherwise, you’re going to feel disconnected – it’s not going to feel natural. The same goes for the environments; they are not static, from another virtual place. We’ve captured real-life environments, and brought them into VR. You really feel like you’re in the dynamic, living places.

TC: Who is this for?

JL:  We are aiming this for consumers. It is for friends who are connecting. Families who want to get connected. The solution is tied to a login rather than to the physical device, so that unlocks the entire household if they have a device in the house to be used by multiple people. Of course, you can use it for work, but the applications beyond just catching up and talking with someone are huge. There are a plethora of possibilities, not only in the business applications but in coaching applications, for example, or in the medical world. We are excited to see where our consumers can take it.

TC: How are you planning to monetize this?

JL: We are announcing this at CES, and it will be available next month. For now, it will be a free-to-access solution. In the future, we are going to be coming out with some premium features and some premium locations, but there will always be that free underlying tier that people will be able to access. Our goal is to get people to communicate in this new way.

TC: What about rich, immersive experiences? Can I show someone a picture?

JL: As people are communicating, the first thing that might come up is wanting to show their favorite cat or dog videos. A baby’s first steps, things like that. With your mobile device you can upload personal media and share that in the call, as well. You can have visuals with you, as well. Here, too, we are leveraging our imaging excellence. We wanted to do more than just say here’s a photo. You can drag the image or the video around the environment, and they can stick as well. Imagine yourself making the room and the space your own. The environments may be the same, but just like you put wallpaper on your phone to personalize it, you can add custom pictures to your Kokomo environments.

TC: What has it been like to build this as an intrapreneur within Canon?

JL: As I mentioned, this is a new direction for Canon. That is true publicly, but also internally. It has all been very new for Canon. Ray (Konno), Jason (Williams) and myself brought our areas of expertise into this to help build this project. We engaged our development group and really leaned on our expertise. We have people working on focus, depth, color capability, and now we are working in very new ways from working on new hardware products. That means we’ve had to work in new ways from an organizational standpoint, as well. This project is not just U.S.-based, either – it will also be releasing in Canada. In the future, we are planning to go global, working with all of the Canon groups, including Canon Inc.

TC: Where do you hope this will go? 

JL: Really, this is a new communication style; it is really like being there in the place. When people leave a conversation, they refer to using Kokomo as “being with” a person, rather than “talking with” a person. We hope that this will continue to evolve.

Ray Konno (RK): What we were seeing was that VR has been becoming very common. We foresee that communication is having another step forward. Video calling became so common, but when it comes to more exotic technology, we see more potential in the next steps of communication. We’ve been supporting the emerging communication trends from the Canon side. I see a lot of potential in the way that people connect, especially when there is physical distance. It feels like you feel when you meet face to face. What we prioritize is to make this technology accessible without needing a lot of gear and expensive hardware. That’s why our whole configuration is super simple. You only need a mobile phone and a VR headset.

Jason Williams (JW): Our mission is to enable people to really be there. To achieve that, our objective is really to enrich and facilitate an engaging interaction with another person when we’re in Kokomo. So our goal as we look down the road is to continue to build upon that. How do we enrich that engagement? How do we make it feel more compelling? The steps we’ve taken already include the ability to bring in a media player, and get together with our friends. From there, we share our photos and videos, and we engage and we talk. Of course, we’re looking to expand the ways that we interact with one another with this type of technology.

JL: I used to be at Disney, and at its core, Disney is an entertainment company. At its core, Apple is a people company that looks to enrich lives. We are looking to bring people closer with all of our technologies. That’s really what we do, whether on the business side, on the consumer side, and across the company.

TC: What does success look like for you? 

JL: For now, the biggest thing for us is that people download it, start to use it and start playing with it. To me, success is not just one call. It’s people saying “I want to do that again! And now I want to call someone else!” Our mission is not to sell headsets or to sell mobile devices. Our mission is just to get people using the software and saying ‘wow, this is great.’ And then from that, we can explore the specific niches where this can be used. “Oh I could use it this way” or “Oh! I could use it for that!” This is just the beginning. We imagine a future with immersive films, and immersive stories, so that when you’re with someone you could jump into something you could experience something together. And that in itself can innovate entertainment and entertain in new ways, also in the future.

JL: Media will be controlled by the mobile device, so you can tap on your camera roll to share. It will then upload, and once you’re in the headset, it’s as simple as just looking at the palm of your hand, which brings up a neat little menu. Here, you can click on the media library and then you can tap and drag. It makes the experience really seamless and easy to use.