In November 2021, Facebook announced it would delete face recognition data extracted from images of more than 1 billion people and stop offering to automatically tag people in photos and videos. Luke Stark, an assistant professor at Western University, in Canada, told WIRED at the time that he considered the policy change a PR tactic because the company’s VR push would likely lead to the expanded collection of physiological data and raise new privacy concerns.
This week, Stark’s prediction proved right. Meta, as the company that built Facebook is now called, introduced its latest VR headset, the Quest Pro. The new model adds a set of five inward-facing cameras that watch a person’s face to track eye movements and facial expressions, allowing an avatar to reflect their expressions, smiling, winking, or raising an eyebrow in real time. The headset also has five exterior cameras that will in the future help give avatars legs that copy a person’s movements in the real world.
After Meta’s presentation, Stark said the outcome was predictable. And he suspects that the default “off” setting for face tracking won’t last long. “It’s been clear for some years that animated avatars are acting as privacy loss leaders,” he said. “This data is far more granular and far more personal than an image of a face in the photograph.”
At the event announcing the new headset, Mark Zuckerberg, Meta’s CEO, described the intimate new data collection as a necessary part of his vision for virtual reality. “When we communicate, all our nonverbal expressions and gestures are often even more important than what we say, and the way we connect virtually needs to reflect that too,” he said.
Zuckerberg also said the Quest Pro’s internal cameras, combined with cameras in its controllers, would power photorealistic avatars that look more like a real person and less like a cartoon. No timeline was offered for that feature’s release. A VR selfie of Zuckerberg’s cartoonish avatar that he later admitted was “basic” became a meme this summer, prompting Meta to announce changes to its avatars.
Companies including Amazon and various research projects have previously used conventional photos of faces to try to predict a person’s emotional state, despite a lack of evidence that the technology can work. Data from Meta’s new headset might provide a fresh way to infer a person’s interests or reactions to content. The company is experimenting with shopping in virtual reality and has filed patents envisioning personalized ads in the metaverse, as well as media content that adapts in response to a person’s facial expressions.
In a briefing with journalists last week, Meta product manager Nick Ontiveros said the company doesn’t use that information to predict emotions. Raw images and pictures used to power these features are stored on the headset, processed locally on the device, and deleted after processing, Meta says. Eye-tracking and facial-expression privacy notices the company published this week state that although raw images get deleted, insights gleaned from those images may be processed and stored on Meta servers.
