Whether you love it or hate it, you have to agree that Alphabet’s Niantic Inc. (formerly known as Niantic Labs under Google) managed to hit gold with its ideas of exploration and real-world social interaction using our mobile smartphones to deliver Augmented Reality experiences. These AR experiences gave the average Joe another reason to go outside, explore, and meet people with similar interests, all the while engaging himself with content that changed with his surroundings.
While Ingress was a modest success, it was Pokémon GO that catapulted Niantic and the term “AR” into viral popularity. Extensive public interest into these emerging technologies allowed Niantic to raise $200 Million in Series B funding in 2017 and spiked its valuation to $2 Billion, which then opened up opportunities to acquire startups like Escher Reality which built backend services for cross-platform mobile AR that relied on shared and persistent AR experiences (much like how apps like Just a Line utilize Google’s ARCore work). The company also announced its collaboration with Warner Bros. to create its next augmented reality mobile game — Harry Potter: Wizards Unite, trying to leverage the brand popularity of the Harry Potter franchise in a fashion similar to that of Pokémon GO.
Today, Niantic is making two important announcements. Firstly, the company has acquired Matrix Hill, a company working primarily on computer vision and machine learning. These new employees will also work out of Niantic’s new London office.
Secondly, Niantic also unveiled its AR platform. Called “The Niantic Real World Platform,” this AR engine is what powers the experiences on Pokémon GO, using a cutting-edge blend of computer vision, depth detection, and real-time object recognition. All of this also needs to be done on power-limited mobile devices, which adds a severe element of restriction.
The Niantic Real World Platform is building towards contextual computer vision, where AR objects understand and interact with real-world objects in unique ways–stopping in front of them, running past them, or maybe even jumping into them.
Utilizing computer vision and deep learning, Niantic has been able to understand 3D space much better, enabling more realistic AR interactions than what is currently possible. The company demoed this in a short video, showcasing how AR objects interact with the reality, seamlessly blending around real-world objects.
Niantic’s next challenge area is bringing this AR experience to a multiplayer gameplay format by tackling issues of latency. Components of Niantic’s Real World Platform must work in tandem across multiple users with varying perspectives in order to create a shared, multiplayer experience that is visually appealing and real-time. Niantic addressed this latency issue by developing a proprietary, low-latency AR networking technique, which in turn helped them build a unified, cross-platform solution that enables this shared AR experience with a single code base.
This low-latency AR networking technique removes the need for a smartphone to communicate with a server before establishing a connection to a nearby user. Instead, the network of devices lets each one communicate directly with another through cell tower transmission, allowing for lower latency connections and more immediate interactions with other players.
While Niantic’s developments in the field of geospatial, real-time, and shared AR are definitely exciting, what is even more exciting is their decision to open up this platform for other developers to build on. However, Niantic is keeping its silence on how exactly they plan to allow access to their Real World Platform and any API capabilities, and how this would tie into revenue generation for the company.
“We will be selecting a handful of third-party developers to begin working with these tools later this year, and you can sign up to receive more information here.”
These developments also shed light on what could be coming to existing Niantic products. Ingress is gettings its own “massive overhaul” this year in the form of Ingress Prime, featuring a new look, a new story, and hopefully some of these new technologies. Ingress’s Operation Portal Recon initiative has also managed to solve Niantic’s problems with community submissions of Points of Interests (landmarks that are adopted into Ingress as portals and into Pokémon GO as pokéstops and gyms) by leveraging the very same community to crowd-approve new places.