How Apple ’tuned’ the iPhone X for Augmented Reality

A lot of small but very important things come together to make the iPhone X great for Augmented Reality.

With a simple software update, iOS 11 will enable a universe of Augmented Reality experiences on millions of iPhones. That's going to add up to a lot of very cool experiences very quickly, but are all of those experiences going to be the same? During the iPhone X presentation, Apple's Phil Schiller said this future-forward iPhone was "specially tuned" to deliver the best Augmented Reality experiences. What does that mean, and how different will Augmented Reality be on this new iPhone?

Here are a few important details that will help you understand what is going on here.

The camera placement matters a lot

The dual-camera set up on previous iPhones has been horizontal, and that makes sense when a lot of the things you're going to be doing with those cameras happen while the phone is in "portrait" orientation. On the iPhone X, these cameras have moved to a vertical strip with the flash in between the sensors instead of off to the side. This may seem like a small detail, but the unique positioning of the camera on the iPhone X has a lot to do with Augmented Reality and how well ARKit apps will perform. Most ARKit apps are going to be used while the phone is in "landscape" orientation, and these cameras are positioned to take advantage of this.

By having the cameras horizontally aligned in the landscape orientation, but also spaced a little further apart, Apple can use these cameras more effectively as depth sensing tools. Better depth sensing means better spatial awareness, which in turn means more accurate measurements of the surfaces being used to place Augmented Reality objects.

The difference in camera placement is going to be a subtle thing when the iPhone X is first launched, but Apple will continue to improve this experience over time. The more information Apple has from developers on how to improve ARKit, the more this camera array will stand out when compared to how well the other iPhones handle ARKit apps.

New gyroscope and accelerometer

This little detail in Phil Schiller's part of the iPhone X presentation was quickly presented without a lot of explanation or detail, but it's a hugely important detail for Augmented Reality and contributes a lot to what it means to be "tuned" for ARKit.

these new sensors are going to be even more precise than the excellent sensors already in the iPhone 7 and iPhone 8.

ARKit works by placing an object on a surface and allowing you to move around it as though that virtual object was really there in the real world. A big part of making that experience feel real is making sure the phone never "loses" the position of that object. To ensure this all feels like magic, the gyroscope and accelerometers are used to determine how you have moved in relation to that virtual object. If you move closer to see more detail or walk around the object, quality motion tracking will make sure it feels like that thing is really there.

Apple clearly felt it was important to upgrade the accelerometer and gyroscope in the iPhone X, which likely means these new sensors are going to be even more precise than the excellent sensors already in the iPhone 7 and iPhone 8. That accuracy bump, however subtle it may seem when comparing these phones, is likely to have long-term benefits we're not able to see yet.

The OLED display pulls it all together

Apple is tremendously proud of its display quality, and for good reason. The switch to OLED for Apple has been a long time coming, and it looks like this display in particular is set to really wow folks. The use of HDR and True Tone on the iPhone X display means it will truly look like you are peering through a window into an alternate world when using ARKit, basically anywhere. OLED displays are capable of spectacular brightness outdoors, and that is going to make a big difference in places where the LCD-based iPhone displays might not perform quite as well with AR.

More than just the ability to enjoy the phone better outdoors, this nearly bezel-free design is going to make the phone a lot of fun to use for AR apps. You still have to hold the phone up to your face, so you've never going to really feel like the phone is gone, but phone designs like this make immersion a lot easier to accomplish.

Get excited for Augmented Reality on the iPhone X

Neither of these individual features are going to make a massive change in how you use this phone or the ARKit apps contained within, but together these features are going to make a difference. The display and the sensors and the cameras are all going to come together to create an AR experience that can't be achieved anywhere else, including other iPhones.

Being "tuned" for Augmented Reality makes it clear from the beginning that this is the phone Apple has in mind when building Augmented Reality features. That means, more than anything else, if you care about AR as a feature on your phone this is going to be the iPhone you want in your pocket.