In an unusual exercise, Meta (the parent company of Facebook, Instagram and WhatsApp) recently opened the doors to its R&D labs. During a video conference hosted by Mark Zuckerberg personally with some of the group’s engineers, the social networking giant presented a series of prototypes of virtual reality helmets – each more futuristic than the other.
The exercise comes as investors worry assets gobbled up by the Menlo Park group to develop metaverse technologies, a hyper-realistic digital universe on which Meta relies for its future growth. The group’s Reality Labs division lost $10 billion last year. And in the first quarter, the group took out an additional 3 billion there, for barely 700 million in revenue… Just at the moment when the group’s growth is slowing down significantly, Meta needs to calm the credibility of its bet.
“Virtual Turing Test”
He is extremely ambitious. The objective of the R&D teams is to create display systems on par with human vision. Inside, engineers talk about the “virtual Turing test”—referring to the famous test created in 1950 to assess a computer program’s ability to pass for a human being. Similarly, Meta wants its headset users to be unable to tell whether they’re looking at a screen or the reality of the outside world.
“It’s much more complex than displaying a realistic image on a computer or TV screen,” says Mark Zuckerberg. Human vision involves many parameters that must be reproduced if we want to trap the user’s eye and brain. Therefore, the Meta teams have developed several prototypes – each trying to solve one of the many challenges of the “virtual Turing test”.
Resolution and varifocal
The first device Mark Zuckerberg pulls out of his bag is codenamed “Butterscotch.” It aims to drastically increase the number of pixels on the screen. Ideally, it should have 8K resolution, across the VR headset’s field of view – much wider than a TV screen.
The Meta teams have managed to double the resolution of the Quest 2 (helmet currently on sale), to approach that of the human retina. “We can line up the 20/20 line of an optometric chart [ceux utilisés par les ophtalmologues, NDLR] “, rejoices Mark Zuckerberg. But the engineers had to sacrifice half the field of view for this, and the device remains bulky and heavy.
The second prototype is called “Half Dome” and addresses the issue of depth of field. This is fixed in helmets currently on the market. But “in the real world, the lens of our eyes acts like a lens and constantly changes shape to focus on what we’re looking at,” says Mark Zuckerberg. After experimenting with mechanical variable focus display systems in 2017, Meta switched to an electronic version, which is simpler and more robust. Thus, the helmet is able to follow the gaze and adjust the depth of field according to where it is worn – a bit like the automatic focus of a camera.
A lens hologram
A third model, Starburst, wants to offer brightness and contrasts as good as natural light – which means multiplying them by a hundred compared to what current helmets do. The prototype is so heavy it has handles.
To solve the main concerns of mass and weight, Meta has one last toy: the Holocake, a helmet that looks more like a simple ski mask, in which light does not pass through a lens, but through a hologram of a lens. The only problem with this holographic system is that it would require a reliable and above all cheap laser to power it.
Eventually, all these solutions will have to be integrated into a single device. Meta has also modeled this helmet of the future, called “Lake of Reflection”. “Nothing in physics is stopping us from getting there,” boasts Mark Zuckerberg — even if he’s wary of setting a date. Not to mention that in addition to the screen, it will be necessary to solve many other problems at the level of sensors, computing power on board, battery…
The first devices to benefit from Meta Labs’ innovations will be, according to Mark Zuckerberg, probably too expensive for the general public and reserved for professionals. This will already be the case with a helmet called “Project Cambria”, due later this year, which is aimed at companies and will be the first to deploy a vision and face tracking system in order to allow avatars make eye contact and communicate with facial expressions. . One small step for the metaverse, one giant leap for Facebook engineers.