Sony’s Upcoming MR Headset Could Point the Way for Controllers on Vision Pro

Vision Pro doesn’t have controllers, but it’s clear Apple will need to support immersive hardware input somehow in the future, if only for the greater precision it might bring to industry professionals. Sony has a pretty interesting take on how to serve up ‘pro’ style controllers in its recently revealed enterprise-focused standalone, which could point the way forward for Vision Pro.

Vision Pro is a mixed reality headset with a bit of an identity crisis. While it can technically play fully-immersive VR content, without motion controllers many of the VR games you’ll find on Quest simply won’t make the jump to the $3,500 device, which is instead being pitched as a productivity machine capable of running the standard fleet of iOS apps in addition to visionOS-native apps.

Ok, maybe not an “identity crisis” as such, but Apple made some clear marketing decisions with Vision Pro by omitting controllers, instead focusing on eye and hand-tracking input. Granted, that’s not to say we’ll never see motion controllers on Vision Pro, but probably not the all-purpose Touch-style controllers many manufacturers have converged upon following Meta’s dominance in VR. At least not in the near future.

At CES 2024 in January, Sony unveiled its Snapdragon-based standalone MR headset (still unnamed) which promises a ton of enterprise-friendly features, like “4K” OLED microdisplays, passthrough video, a flip-up visor, and a unique pair of controllers that aim to appeal to the sort of professionals Apple undoubtedly wants to capture at some point too.

When it ships sometime “later in 2024,” Sony is including both a pointer-like controller that’s operated with the index finger, and a small ring-like device that’s controlled with the thumb. Yes, they’re funky, but they have some pretty specific use-cases in mind.

The ring lets you select items and anchor them firmly to your hand, while the so-called ‘pointing controller’ enables what Sony calls “stable and accurate pointing in virtual spaces, with optimized shape and button layouts for efficient and precise operation.” It’s basically a 3D stylus Sony hopes will appeal to creators building and viewing 3D objects in the headset itself, but in a natural and precise way, making for an ostensibly pared down version of Wacom’s VR Pen, which was announced back in 2020.

Wait. Why are controllers so important in the first place? Vision Pro’s hand-tracking is some of the best—and will likely evolve over the headset’s life cycle—but all of the same caveats apply to Vision Pro’s optical hand-tracking as they do in any modern XR headset. That’s to say, there is only so much Apple (or anyone) can do without controllers.

Hand-tracking doesn’t provide haptic feedback, which isn’t great for establishing physical relationships to digital objects. Most of all though, using an empty hand to ‘pinch and drag’ doesn’t provide the sort of precision grip you can get from tracked controllers, which necessarily include buttons that don’t need to ‘guess’ when you’re using them, since buttons have precise activation thresholds that you can intuitively learn to manipulate and anticipate. The button ‘knows’ when you’re just resting your finger on it, when you’re depressing it half-way, and when it’s all the way down—all of which can be integrated into a more rich and accurate way of spatially computing beyond just browsing through UI menus and doing some lighter gaming stuff.

Not to mention, going for something similar to Sony’s ring/pointer combo would allow Apple to retain that pro-level marketing bend, and also offer a sleek (and very likely expensive) accessory we all know Apple loves to sell—but that’s besides the point.

It’s not like any of this is a big surprise to Apple though, or the developers making visionOS-native apps that use of the headset’s 3D spatial abilities. For now, Apple wants Vision Pro to be more ‘face-computer’ than gaming console, but also more ‘bottom-tier iPad’ than Mac Pro, at least in terms of hardcore industry apps it can support. There’s plenty of room to grow however, as right now Vision Pro has a few apps for viewing 3D assets in immersive space, like Polycam and the enterprise-focused app JigSpace, and casual sculpting and 2D art apps—but still none of the industry standard tools like Blender or Autodesk, both of which actually support VR headsets.

Still, Apple is very much a ‘wait and see’ sort of company, so its competitors may need to collectively be “the first guy through the wall” (who always gets bloody) before the Fruit Company follows suit. Given a second iteration of the headset has been rumored to arrive sometime in the next two years, Apple may be waiting to bump that particular feature set later rather than sooner, if only to have that really useful One More Thing™ to announce next time Tim Cook takes the stage.