Apple’s new iPhone is much more about AR than you think

Some commentators have dismissed the processing speed upgrade of the iPhone XS and XR as being fairly minor. But the effect such extra processing power could have on mobile AR may be huge, and it's a key part of Apple's current strategy.

Last week, Apple hosted a special event to unveil three new iPhones: the iPhone XS, XS Max, and XR (ten-R). Although--much to my disappointment--the iPhone XR has nothing to do with mixed reality (a combination of augmented and virtual reality often referred to as XR), there was still news that should interest immersive content creators.

Apple’s new mobile operating system, iOS 12, which launched on Monday, and ARKit 2, Apple’s augmented reality development platform, will be a major part of the update.

ARKit 2, first announced at WWDC in June, offers improved face tracking, more realistic rendering, 3D object detection support, and enables shared AR experiences.

Apple showed off one such shared AR experience at its special event yesterday, in the form of playing Galaga in augmented reality alongside friends.

They also showed off another AR-based app that measures trajectory, release times and heights and other information to track basketball shots.

Hall of Fame basketball player Steve Nash and CEO of HomeCourt David Lee explain the real-time shot science and machine learning of the HomeCourt app. Photo courtesy of Apple.

Quick Look

Easily the most impressive augmented reality feature Apple announced was a new feature called “Quick Look”.

Quick Look enables users to pull a 3D object out of a web page in Safari and place it in the real world, via AR. This means developers can utilize augmented reality without building a new app. Instead, they will be able to integrate web-based AR into their sites.

Shopify, for example, has already announced its plans to integrate Quick Look into its online retail platform. The feature will also work for other Apple apps, like Messages and Mail.

Also with iOS 12, Apple has launched a new utility app called Measure, which can measure the dimensions of real-world objects using the iPhone’s camera.

iOS 12 also makes it possible to have persistent experiences, which means AR apps will be able to capture AR experiences at a certain time or location for others using the app at that time or location to experience later--a useful feature for tours or art exhibits.

Not to mention improvements to Apple’s animojis, including improved face tracking, the addition of new animojis, an improved interface, and the ability to make your own ‘memoji’ (similar to Bitmojis in SnapChat).

CEO of Directive Games Atli Mar Sveinsson demos the gameplay of Galaga AR with three players. Photo courtesy of Apple.

Huge benefits

Although iOS 12 launched on Monday for iPhones 5S and above, the new generation of iPhones feature an A12 Bionic chip that offers a number of benefits for augmented reality, including 9x faster processing speeds, reduced power consumption, and real-time machine learning.

Although every iPhone owner will benefit from iOS 12, in terms of augmented reality, those who stand to gain the most will be those of us who invest $749 for the iPhone XR, $999 for the XS, or $1099 for the XS Max.

Apple’s Tim Cook has been keen on augmented reality for quite some time, saying last October that he thought AR’s impact would be “profound”.

“I’m incredibly excited by AR because I can see uses for it everywhere,” he’d said. “I can see uses for it in education, in consumers, in entertainment, in sports. I can see it in every business that I know anything about.”

And, at the end of August, Apple acquired Akonia Holographics, a startup making lenses for augmented reality glasses, offering further proof that a wearable AR device from Apple may be in the works (some rumors have estimated one would be announced as early as 2019).

Regardless of the launch of an AR wearable, at the very least ARKit, iOS 12, and the A12 Bionic chip offer proof that Apple still believes augmented reality will be profound.

The next generation of iPhones includes the XS, XS Max, and XR (seen here). Photo courtesy of Apple.