Apple Vision Pro: disappointment that will be good for the future of xR

There are already many reviews of Apple Vision Pro, but I’ll try to give it a different perspective as someone who is not a journalist or an Apple fanboy, and cover things that for some reason other reviewers didn’t mention. It is based on years xR experience and 40 minute demo at Apple store.

I’m not an Apple fan, nor Microsoft/Meta/Google fan, and I’ve used used pretty much all xR headsets and platforms, many 3D/360/VR180 cameras so I will present here everything from that point of view, and fill-in the gaps of other reviews. Some parts might have jargon from those industries but I’ll try to make it non-technical where possible. The last time I wrote about these topics was when Windows Mixed Reality (later mentioned) came out, and I used it in a NYC subway and shared my experience.

I’ll try to answer what Apple brings to the table, is there difference between the “Spatial computing” and “xR” headsets on the market, and where it falls short.

Demo at the Apple store

I booked demo at Apple Store on Grand Central in NYC. It was a 40-minute demo with one employee dedicated just to my experience. It didn’t go 100% smooth, but overall, the experience was a lot better than when I demoed Microsoft WindowsMR headsets on NYC Microsoft 5th avenue store, where they couldn’t even show it to me, because they were installing drivers for the entire hour, it got stuck at half-way through an install, and then told me to come the next day. It didn’t work out the next day either. Story for another time.

At the Apple Store, I told the person leading the demo that I had used pretty much all “headsets” on the market, but they explained that Apple doesn’t describe Vision Pro as a “headset” but a spatial computer, “something completely new”. I’ll return to that.

I’ll call it a headset, because it is worn on the head, has a display, cameras for seeing/tracking the environment, audio, and all other elements that other devices we call “headsets” also have. The Vision Pro is essentially both a VR (Virtual Reality) and AR (Augmented Reality) device, with a pretty nice way to transition between those, or we can call it XR/xR (extended reality, or “x” being a wildcard for “V” or “A”) or MR (Mixed Reality).

Apple will call it “spatial”, because they want to be seen as unique. Remember “Mac vs PC” ads in the early 2000s? They claimed that “Mac is not a PC”, which is, by the way, short for “Personal Computer”, so if it is not a personal computer, what is it then?

Today this is the same insincere advertising, but they can pull it off, unlike Microsoft which tried inventing term “MR = Mixed Reality” for their VR headsets, and then alienated even the “PC” users who thought it was not the VR (WindowsMR headsets were purely VR headsets, not really “mixing” realities). Then they invented “immersive” MR headsets that were supposed to be “VR”… which would be a long article which I won’t tangent into.

First impressions

Build quality is good (as it has to be for $4,000 including tax). It doesn’t look like a toy (it doesn’t crackle under fingers like HTC Vive Pro did). While this is good, I think Apple shot itself in the foot, as they have set the bar very high and they can’t just make a plastic headset called Vision SE… actually, they can and people will buy it)

The Vision Pro looked good there “spatially” floating on a metal stick. Constantly polished by store employees, looking shiny. Most people don’t realize that as soon as you take it home or your office… the glass on the front will look as bad as Meta Quest Pro does in the real-life use, covered with fingerprints and smudges. Glowing display will surely help hiding the smudges on the reflective front.

While I was sitting there chatting with my demo guide, "my" Vision Pro unit was brought on a platter by the second employee, carrying it with two hands like a dish in an expensive restaurant.

I put it on…

Visual clarity (resolution and dynamic range) are great, the best on the market. Text looks sharp and it could be used for reading - although I’m pretty sure most people will choose to read a 15-minute-long text on an actual flat screen instead. Not because the text will still look better on a proper tablet or a computer screen, but because of the unnecessary neck strain and no benefit of the “spatial” aspect for just reading text.

Another thing I noticed there, that nobody else talks about, is that moving the head causes a slight motion blur, which is noticeable the most on the high contrast edges of text, so you have to keep your head perfectly still, as even a small breathing motion makes the motion blur apparent (I am not talking about camera pass-through blur).

Camera/Video-Passthrough resolution was decent, noise acceptable, not great but still the best quality on the market today. I don’t put much importance on it, as nobody will be buying it to look the “real-reality” through it. I have to mention that you can’t wear it with eyeglasses (like you can other headsets), and you have to buy prescription inserts (which are also available for other headsets). I still prefer how some other headset manufacturers provide just a dial to set your diopter.

This just means that there is no sharing of this headset unless everyone has the same diopter (or wears contact lenses).

Field-of-view on Vision Pro is the lowest on the market, even lower than $200 WindowsMR (VR) headsets we had 2017. The Field-of-view is basically how wide you see — imagine looking through binoculars, you see two circular images in the darkness, and on Vision Pro those circles are slightly smaller.

Weight — front heavy because it is made of aluminum. They didn’t have a top strap for demo (sold separately?), which would definitely help distribute weight to the top of the head instead of my face. If felt heavier than Meta Quest 3 with the stock strap, but Meta Quest Pro was still more comfortable (and most headsets with rigid “halo” type straps) or aftermarket straps we can get for $30 on Amazon. Twitter is already full of people buying $4000 Vision Pro and then zip-tie it to a $30 aftermarket strap made for Meta Quest to make the Vision Pro usable.

Size — I was under impression it is shallower but it goes well beyond the tip of your nose. I thought this Chinese knockoff I saw on CES 2024 was funny because of being very deep, the Vision Pro is not much shallower.

Comfort/Outside light blocking — good. They explained that they have many different face cushions. They scan your face with an iPhone and give you one that is the closest to the shape of your face — which means that for multi-user environments/sharing it won’t work very well. In addition, due to the headset making a full-face contact, sharing also means spreading bacteria between each user. To compare, Meta Quest Pro is not even touching your face and I've been using halo-type head-straps since the beginning so I am not used to headsets that are resting on my nose/face.

Onboarding experience — simple, especially compared to all other headsets on the market.

Interaction with eye-tracking to highlight, and pinch to select would be effortless, if it was more precise. Sometimes looking at an element didn’t highlight it, so I had to look at random corners of the element until it got highlighted. Maybe repeated calibration would help. This also means that you can’t just share it with your spouse and expect it will work for them. That is another issue for any multi-user environment, because calibrating it between each user would be a hassle. Although, I am not sure if this is a good idea at all as you can’t just let your eyes naturally wonder, but require precise hand-eye coordination, which will become strenuous.

This is not a new interaction or technology. I used a similar eye tracking as input on Playstation’s PSVR2 (just for demo) so I can’t compare which one is better, just that it is not new. Some laptops also had eye tracking for even decade ago.

These interactions are easier than on Meta Quest 3 (which doesn’t have eye tracking) where you have to use a ray going from your hand (or use a spatial/motion controller) to point to an element before pinching to select. But at least with Meta Quest 3 you can give it to another person and they can use it without needing to calibrate the eye-tracking. The Vision Pro is a personal device (even though the price suggests that costs should be shared among a few people), but Apple probably doesn’t care about this.

Motion blur — pixel persistence issue

This is my one of my biggest complaints, and I don’t know why nobody else talks about it. When you move your head, the entire view blurs (pixel-persistence issue, reminding me of a faint ghosting that we had on Lenovo Explorer headset). I am not talking about pass-through motion blur from the cameras looking at the real-world (also there), but the rendered content suffers from this the motion blur on the head motion. That might be OK for the intended group since this headset is not meant for action games or moving your head much. You wont notice it watching Apple TV with your head just pointing forward.

Content from the demos

“Panoramas were useless, until now” said my demo guide, showing a panorama taken with an iPhone. Maybe yes, because getting a panorama from any phone into a VR headset was a few clicks to many to do it more than 3 times. But, it was possible, and these iPhone panoramas are far from the best. That iPhone panorama was about 250° wide and about 70° degrees tall. There have been many 360 panoramic cameras (360° wide and 180° tall) on the market for a long time. You can find 360 photos taken with those cameras by users all over the Google Maps (one of mine from 2017), and I am not talking about the broken patchwork from Google phones from Google Nexus 4 days). My favorite 360 camera was Ricoh Theta S (back in the day of 2015), Samsung had Gear 360 camera, there were other weird consumer VR cameras.

Since then we’ve all got tired of panoramas and moved to VR180 photos in stereoscopic 3D, or as Apple calls them…

Spatial photos. They demoed one taken by a Vision Pro — impressive, but we’ve had it for decades. Most Apple users won’t realize that those photos were possible and looked as good even over a decade ago. FinePix Real 3D W3 was released in 2010 (and as you can guess from W“3”, it wasn’t the first one). That camera had a better 3D effect than the grainy 3D spatial photo taken with Vision Pro that I’ve seen on the demo (take a closer look). W3 photos were actually a proper 3D because it had two lenses (with a similar distance as between the eyes), while iPhone’s spatial photos are just a faked 3D from a single photo and depth camera information, so it looks somewhat flat. Between release of that camera and today, we’ve had VuzeXR camera (now discontinued) and even Canon released an expensive lens for Canon R5 for shooting s̶p̶a̶t̶i̶a̶l̶ VR content. These, and many more are making 180°x180° degree true 3D photos— unlike Vision Pro which I’d guess were closer to 60°x60° photos. Those are still better than flat photos but don’t make you feel as if you are there (the captured scene has to be larger than the headset’s field of view).

Spatial videos, again impressive for people seeing it for the first time, but non-Apple users had VR or spatial videos available for many years. This even includes adult content from probably over 30 studios making VR adult first-person “spatial” videos. The demoed concerts, basketball/baseball game, and similar are captured by the cameras like the mentioned Canon R5 or Z-Cam K1 Pro and this is not something that Apple “invented” or produced.

They showed child’s birthday recorded by person wearing Vision Pro. It looks like you were “there”, but let’s be honest, that person recording this was not there more than you were looking at it. That person had the same screen-mediated experience like you did, and my guess is that your kid wouldn’t be as excited if her dad was “present” at her birthday party wearing ski goggles. Here is the still from their promo video (not the one they demo):

3D movies on Apple TV. Sounds like fun. I’ve watched a few in VR headsets over the years, like Ready Player One, Avatar, and similar (yes, it is exactly the same experience as on Vision Pro and you can even choose a movie theater as the environment instead of your home). You will watch a few of those 3D movies too, and then, if you have a spouse, realize that the same movie is more fun when watched together on a TV even if is flat.

That reminded me again on Apple’s announcement video for Vision Pro. It looks like a lonely woman, maybe a lawyer (she has $4000 for it), after 12 hours at work comes home, sits alone and over a glass of wine watches a film. Compare this to target audience of Meta with young folks going active after work and play Beat Saber instead. Notice passive vs active consumption.

But the place where Apple will shine is in a way this passive content will be brought to the user (well, probably they won’t allow those adult “spatial” videos). There is a ton of VR players we’ve had for years, but UX of all to those was bad. I even had my own VR photo/video player in development about 5 years ago and I abandoned the project. I don’t have my own AppleTV/YouTube company to serve the content and not many people had VR cameras back then to take advantage of it (and because state of VR frameworks/toolkits was (is?) a mess).

I don’t have much to say about the reverse-passthrough (seeing user’s eyes in front of the Vision Pro) because they didn’t demo that, but as someone who experimented with lenticulars and autostereoscopic displays, I know what we can expect from it, and I consider it a gimmick that just wastes the battery and processing power (plus it looks bad with eyes where your eyebrows are, and low resolution)

Conclusions

Overall, the demo was OK. All of us who have used headsets in the past 7 years have already seen it all, and it is still interesting, just in slightly higher resolution presented by Apple. The virtual view used be like looking through a screen-door/mosquito-net back on Oculus CV1 headset from 2017, but everyone has stepped up their game, and since Apple is not producing those display themselves, everyone will be able to source the same displays (but hopefully without the motion blur issues of Vision Pro mentioned before).

I don’t watch 3D movies on it anymore. The most utility today from VR headset I get from rhythm/exercise games like Beat Saber (yes, still), Ragnarock, or computer games in VR like Half-life Alyx and Skyrim VR when connected to a PC. Those are not even possible on Vision Pro since it doesn’t have controllers. If I am going to put on a headset, I want to stand and move, as I am sitting in front of the computer screens the entire day anyway.

I also use it for 3D modeling/designing in VR/AR, again something not possible on Vision Pro since it doesn’t have controllers.

What Vision Pro does well (and is probably designed for) is having multiple flat apps floating in space. It is simple and was done well, unlike the similar app pinning thing Microsoft had on HoloLens.

So with the Vision Pro, Apple is actually selling the same thousands of existing iPad apps that can also run on it as well. Plus they now have an additional device on which we could consume Apple TV. They don’t care about having spatial controllers because iPad apps wouldn’t benefit from those, and it doesn’t bring Apple any more money. They do allow 3D content and there will be spatial games, but it is “a drop in a bucket” that Apple will get from selling puzzle games and meditation apps designed specifically for Vision Pro, compared to just selling what they already have for iPads.

During the demo: “There are three interactions: you can pinch, zoom and scroll”. I am not sure which jobs pay you to do just those three. So Vision “Pro” as a “professional”… in which professions exactly? Stock trader needing to see several tickers at the same time. OK, what else?

Since it doesn’t have controllers this also excludes it from use in any creative professions. Creating spatial content is easier with any other headset.

Vision Pro will never be used for work like HoloLens was, simply because Vision Pro or any similar video-passthrough device will never be approved by OSHA (if the battery runs out, or operating system crashes, camera or screen glitches, you are basically blindfolded and maybe in dangerous situations where you have to react).

If you use it just as a monitor for your MacBook, then it didn’t have to cost this much and have the weight of the entire iPad inside. The mentioned WindowsMR devices were sold at $200 at some point because they were just displays with cameras, and the processing was done on the device you connected it to. Those were connected with a single cable to a laptop, but it is not like Vision Pro is without a cable. The battery block is on a cable anyway, so it could have been the computer on the other end of that cable.

No controllers means it is not meant for gamers (unless you just use it as an expensive TV for flat games). The spatial gaming is possible even with just a pinch gesture and I’ve had a great fun with Fragments and RoboRaid on Microsoft HoloLens headset. Having the story happening in my own apartment was one of the best gaming experiences I ever had, but I don’t see Apple making games like these.

The price point excludes it from being used as just an entertainment device (You can buy five 85" 4K TVs for the price of one Vision Pro).

Using it in festivals, art installations, real-estate offices or any shared work environment is also not optimal as the mentioned face cushion is personalized per user, you can't wear with eyeglasses, eye-tracking and interactions won't work at all unless it is calibrated for each user (and there are no controllers).

To return to one of the main points: is it actually a “spatial computer”? No, it is an expensive “spatial iPad”, until it can do something relevant that iPad cannot.

But anyway, xR Technology came far since HTC Vive was released in 2016. Going from a cubic meter of components, to a single backpack, to a standalone wireless headset (Quest3)… back to a headset with a wired dangling battery (Vision Pro). From a pinch with self-intersecting fingers in 2017, to seeing your actual hands and the pinch that works. From ridiculous AR "goggles" with a fanny pack to ski-like goggles tethered to battery pack that people are not ashamed to wear in public.

I am glad that Apple has entered the xR race even if they are trying to confuse everyone and take credit for some of the past 7+ years of the industry progress. It is not exactly what we hoped for, but this will help the entire industry, and I do hope everything will be called “spatial” now, because AR/VR/XR/xR/MR/Holographic/Stereoscopic/Immersive… was a mess.