Making Cities AReady

By Moetsi.

With Sturfee’s AR cloud technology just out of stealth mode, Tech Trends spoke to the company’s CEO, Anil Cheriyadat, and Head of Business Operations & Partnerships, Sheng Huang, about their exciting new tech and what it’s going to take to digitize the world.

Just recently out of stealth mode, Sturfee wants to digitize the world around us with its AR mapping technology Click To Tweet

ANIL

Before starting Sturfee, I worked at the Oak Ridge national lab, which was home to Summit – the fastest computer in the world at the time, working on computer vision and AI algorithms to be used on satellite imagery in real-time. Really, this was the only place in the world you could do that sort of thing then. And it was here that I developed a passion for understanding the world through imagery — and where Sturfee, as an idea, really began.

I developed a passion for understanding the world through imagery, and that's where the idea for Sturfee was born Click To Tweet

SHENG

For me, I was running biz ops at Niantic right at the beginning. I was there when they launched their very first app — the field trip app — which I helped scale from a few thousand users to a few hundred thousand and eventually millions of users. It was incredible to see the uptake and the real potential of location-based AR gaming.

By creating an AR cloud there is the potential to revolutionize the way we interact with the world around us Click To Tweet

Which, of course, we can see from the incredible success of Niantic’s Pokemon Go and Harry Potter: Wizards Unite. But AR doesn’t begin and end with gaming. By creating an AR cloud there is the potential to revolutionize the way we interact with the world around us.

ANIL

Until now, cameras have been successfully trained to recognize human faces, but no one has built a similar technology to recognize physical outdoor locations. Now Sturfee has developed computer vision tech that can recognize places on the ground using images from satellites up in space. This will allow your phone’s camera to instantly recognize its surroundings and for applications to pull 3D building geometry and ground plane data.

Combining satellite imagery with mobile positioning, Sturfee allows developers to create a range of applications that will change the way you use your phone for navigation, tourism, ride-sharing, gaming, outdoor advertising, and marketing.

What astounded me was the accuracy to which modern satellites could create an image of the world. And how quickly too.

Sturfee has developed computer vision tech that can recognize places on the ground using images from satellites up in space Click To Tweet

ANIL

What most people don’t know is that satellite imaging has improved a lot in the last few years. New satellites can now pick up anything bigger than 30cm in the highest resolution and can create an image of the whole world in just 5-7 days in lower resolution. That’s pretty astounding.

What astounded me was the accuracy to which modern satellites could create an image of the world. And how quickly too Click To Tweet

That means that Sturfee’s AR mesh can pinpoint locations to incredible precision, and access a pretty up-to-date map of urban topology — buildings and roads tend not to change too much in 5 days. So with access to such cool satellite imagery, why did they turn their hand to AR?

Combining satellite imagery with mobile positioning, Sturfee allows developers to create a range of applications that will change the way you use your phone Click To Tweet

ANIL

Augmenting the world around us is fascinating. It creates a whole new layer of interaction with everything. You can create a digital overlay of information wherever you go, and what you can do with that is practically limitless. You can leave notes for your friends to discover in live locations, bookmark the world and play basketball on skyscrapers. It’s really cool.

Sturfee is attempting to solve the problem of creating a digital twin of the world at scale Click To Tweet

But this digital overlay is only possible if the real world is being scanned constantly. We need a true representation of the world, as well as the ability to store this map, in order to actually create these next-generation applications.

And it’s here that Sturfee is making a difference. Offering a Visual Positioning System (VPS) cloud service, Sturfee’s AR City platform is a major step towards understanding 3D locations from traditional GPS 2D mappings. And attempts to solve the problem of creating a digital twin of the world at scale.

You can leave notes for your friends to discover in live locations, bookmark the world and play basketball on skyscrapers Click To Tweet

ANIL

Building persistent replicas of large cityscapes is too computationally intensive to do from the “bottom-up” — having phones on the ground scanning their surrounding environments and uploading those scans to the AR Cloud. To solve this, we are coming from the “top-down”, which cuts both processing and financial costs. With our technology, we can create a machine-readable version of San Francisco in just a week, and detect and update any city changes even quicker.

SHENG

Right now Sturfee already powers 15 cities spread around the world, and we can move into a new city, have it scanned and ready in just a week, which is extremely quick. Mike Levine, CEO of HappyGiant, an AR app developer and publisher, worked closely with Sturfee to create this San Francisco demo using the Sturfee VPS platform in order to show off Sturfee’s city scale AR.

But don’t they also want to access bottom-up imagery too?

Right now Sturfee already powers 15 cities spread around the world, and we can move into a new city, have it scanned and ready in just a week Click To Tweet

ANIL

Yes, we definitely want to make sure our 3D meshes of environments are always up to date. That’s why we work with 3rd party providers to provide us with bottom-up imagery from “edge device” scans [an iPhone, for instance]. We combine this information from the two different perspectives into our pipeline to have the most robust digital representation of the world – from top-down, to bottom-up. We just want to be the most robust end-to-end computer vision pipeline.

Devices are simply not able to process a great deal of depth data, and so there’s not much of it Click To Tweet

And that’s always going to be the limit. Simply, satellites will not be able to get inside buildings any time soon and so Sturfee will need to rely on handheld tech to provide hyperlocal data to their machines. But this data is limited. Devices are simply not able to process a great deal of depth data, and so there’s not much of it.

This is where solutions like Moetsi’s Sensor Stream Pipeline come in; making it easier to process depth data by compressing it on device, and processing it off-device. Or 6d.ai, the leaders in crowdsourced 3D mapping using standard smartphones. But this does require a good data connection, something we’re all hoping 5G will improve

This is where solutions like Moetsi’s Sensor Stream Pipeline come in; making it easier to process depth data by compressing it on device, and processing it off-device Click To Tweet

Sheng

For devices to be able to send and receive information to each other really fast on the AR Cloud, the local network needs to be very fast. 5G will really help with this, of course. The information between edge devices will be sent so much quicker, which will really improve things like multi-player gaming experiences.

It’s no surprise then, that Sturfee has partnered with KDDI Corp, Japan’s second-largest cellular operator, to bring their VPS to all major metropolitan areas in the country. And with orders to integrate their tech with smartphones, wearables and 5G services on the horizon, it’s a really exciting time for the company, and to all of us looking forward to playing around in these Augmented Reality Cityscapes. 

Sturfee has partnered with KDDI Corp, Japan’s second-largest cellular operator, to bring their VPS to all major metropolitan areas in the country Click To Tweet

Olenka Polak is the co-founder of Moetsi along with Adam Polak, whose open source Sensor Stream Pipeline makes it easier to process sensor data data off device, without throttling bandwidth. When she’s not working on Moetsi’s open source project, you can find her at AWE nights in Brooklyn, NY, excited about all things XR.

*Answers have been edited for brevity and clarity