I tried Apple Vision Pro and it's far ahead of where I expected
Apple Vision Pro at WWDC 2023.
As I put my right hand on the aluminum casing around the front of the Apple Vision Pro and my left hand on the soft, fabric back and slid it over my face, I thought of the VR experiences I've had in the past and reminded myself to look for the progress and focus on the big picture. That's because when it comes to these augmented reality and virtual reality headsets, the substance has never lived up to the sizzle.
Review: Apple Vision Pro: Fascinating, flawed, and needs to fix 5 things
Thirty minutes later when I finished my tryout of the Vision Pro, I had a thought I've never had after one of these demos: "I wish I could jump back in and do it again. Right now."
Lots of factors made the Vision Pro's headset experience more enjoyable, but let's break down the biggest ones.
I tried out the HTC Vive XR Elite at CES 2023. No photos were allowed in my Apple Vision Pro demo.
1. The eye-tracking interface is smooth and easy to learn
The first thing you do after you put on the Vision Pro and fit it to your head is press the digital crown and calibrate the headset to your eyes. This involves focusing on a few dots that pop up in a circle in front of you. Frankly, after the demo I thought about how important and effective the eye-tracking is for everything you do in the Vision Pro, and I was startled to think about how that simple calibration was all it took to dial it in.
Also: Apple Vision Pro first take: 3 reasons this changes everything
But, I'm getting ahead of myself. After the eye calibration, you calibrate your hands by holding them in front of you and then simply hit the digital crown to jump into the home view, which defaults to a set of app icons that look like an iPad home screen or the Mac Launchpad.
From here, I quickly learned that all I needed to do to launch an app was to look at it and then tap my thumb and forefinger together while my hand was resting in my lap. To scroll to the next page of apps, I simply held my thumb and forefinger together and pulled it to the left or the right or up or down, like I was pulling a piece of string. Closing a window, an app, or an experience involves a quick tap of the thumb and forefinger to bring up controls, looking at the "X" and then tapping the thumb and forefinger again.
Also: Inside VisionOS: 17 things developers need to know right now
Within five to ten minutes, I was rapidly opening and closing apps, scrolling up and down and right to left, selecting things, and moving apps and windows around in the space in front of me. By the end of the demo, I was doing all of this without giving it much thought and with a lot of accuracy and confidence.
This is by far the biggest breakthrough of the Vision Pro and it's clearly something Apple has been working on for a long time, which was confirmed by Sterling Crispin, a former neurotechnology researcher on the Vision Pro team. Getting the interface right opened the door to creating a usable product, but then the content has to deliver experiences to keep this headset from getting stuffed in a drawer where most headsets go to disappear.
Here's a basic example of the interface on the Vision Pro.
2. The quality of the displays keeps you in the experience
After learning the interface of the Vision Pro, the next thing that jumped out at me was how smooth everything was in moving between different experiences and elements. I opened the Photos app and got to see beautiful iPhone photos at a resolution and immersiveness that blows away the iPhone, iPad, Mac and even Apple TV connected to a giant screen. And when I flipped through different photos and videos, the movement was instantaneous and buttery smooth.
Also: Meet your Digital Persona: Apple's Vision Pro users to get real-time animated avatars
The reason behind both of those things -- the resolution and the smoothness -- was technical, of course. The Vision Pro's displays fit 64 pixels into the same space as one pixel on the iPhone. And the new R1 chip inside the Vision Pro streams images in 12 milliseconds, which is the equivalent of eight times faster than an eye can blink.
The result is that the display never skips, lags, or glitches while you're watching content, jumping between apps, or moving around your space. In every other AR and VR experience that I've seen, there are always glitchy moments that take you out of the experience and can even create some motion sickness. I'm definitely vulnerable to motion sickness from these devices and I got no sense of disorientation or discomfort from the Vision Pro demo.
Also: Wear glasses? Apple's already expensive Vision Pro headset will cost you even more
One final note here: The displays are so good that they outstrip the quality of the content and that's likely to be an issue for a while. For example, the demo included beautiful panoramas shot on the iPhone. But because I have an eye for photography, I easily noticed the pixelated parts of the photo where machine learning clumsily tried to fill in some of the gaps in data. Other higher resolution photos captured by professional cameras and some of the three-dimensional images captured by the Vision Pro itself looked phenomenal. Nevertheless, this will likely be an issue for years to come, in the same way the transition from SD to HD or HD to 4K was for TV.
Most of the content Apple showed in the demo -- from Avatar 2 to courtside NBA cameras to inside a studio with Alicia Keyes to clips of mountain climbers in extreme locations -- all looked incredible and showed off the full capabilities of what studios, storytellers, and software developers will be able to create for this platform.
Here's the Vision Pro with its battery pack.
3. The three-dimensional photos and videos are a must-see
The third part of the Vision Pro demo that really impressed me is also the experience that is the most difficult to describe. It involves the three-dimensionalization of photos and videos that you can take from the Vision Pro and then replay in the headset later. When you view these, it brings them to life in a way that greatly exceeds anything we've ever seen in a 3D movie. I believe that's because of the advanced depth mapping that the Vision Pro can do when taking the photos combined with the massive graphics and spatial presentation capabilities available in the Apple headset.
Also: Will Apple's headset signal the beginning of the immersive internet?
See what I mean about this being difficult to describe?
This is a part of the Vision Pro that you'll have to experience in order to fully understand. Fortunately, demos will be coming to Apple Stores next year when the product is released. For now, I'll simply say that photos and videos experienced in this way on the Vision Pro have a level of richness and realism that represents one of the most exciting leaps forward in tech I've seen over the past two decades -- and far more exciting than anything I've seen on two-dimensional 8K displays. I've never been a fan of 3D. This feels like what 3D should have always looked like.
That said, this feature also exposes one of the biggest challenges of the Apple Vision Pro. The photos and videos you want to capture the most are the ones during some of your most important life events -- a family celebration, a memorable trip, a child's birthday, etc. Those are not typically going to be the times when you want to strap a giant headset onto your face to make a recording.
Also: What is ProRes and why does Apple keep mentioning it?
When we get to the 7.0 version of the headset in 2030, that might be a lot less of a problem as the technology shrinks into a smaller and less obtrusive form factor. But in 2024, it's going to be an issue.
Nevertheless, Apple has made breakthroughs with the Vision Pro that will redefine technology, productivity, and entertainment for the next decade. I have a lot more to say about how the product operates, the content Apple has lined up to make it worth using, and especially the ways we could put Vision Pro to work. For now, I'll save those thoughts for future stories.