How the metaverse could disrupt the in-car experience

With all the interest recently in augmented reality (AR) and the metaverse, car companies are looking to bring AR elements into vehicles. An increasing number of in-car infotainment systems, for instance, now have AR integrated as part of their augmented GPS navigation systems, displaying holographic arrows on a live image feed from the front of the car.

Some auto makers are trying to take this adoption of AR a step further by bringing the metaverse into vehicles. Its introduction to the in-car experience could be through interactive holographic windscreens, or by projecting avatars into passenger seats. Although this presents opportunities for entertainment and education, it also opens the door to distractions and even more pervasive advertising.

The use of AR in cars has been limited until recently. The first AR head-up display (HUD) was the Mercedes-Benz user experience (MBUX) infotainment system, produced in the Mercedes A-Class in 2018. Its GPS visibly highlights junction exits and final destinations on the internal displays, along with potential hazards such as potholes, pedestrians, and other vehicles on the road. Since its introduction, other companies including BMW, Continental, and Hyundai have introduced AR into their own infotainment systems.

There are still no commercially available vehicles with AR HUDs directly on windscreens, however, there are a few concept cars that have showcased the technology. WayRay, for example, has developed the Holograktor, described as a ‘metaverse on wheels’ due to its heavy reliance on AR technology. It is an electric ride-hailing concept car that can be driven remotely from an AR pod by a qualified driver, giving the car the feel of autonomy while avoiding the currently immature fully autonomous-driving systems. WayRay founder and CEO Vitaly Ponomarev has said he hopes the car will be released, possibly with a major automaker, by 2025, but has also said the Holograktor could instead be used as an example for other OEMs that want to make similar cars with WayRay’s AR technology.

The car promises a number of entertainment opportunities: Its seats come equipped with joysticks that can be used to play games on the windscreen’s AR HUDs with others in the metaverse, and it even comes preloaded with a Guitar Hero-esque online karaoke game. The advertising opportunities are also there. According to Ponomarev, “The idea is that you can choose Uber Black, Uber SUV, or Uber Holograktor. And if you choose the Holograktor, your ride will be subsidized by sponsored content so that the price will be much lower.”

It’s unclear if consumers will want ads brought so visibly into their journeys. It may be helpful to have some sponsored content on the windscreens if you are actively looking for a restaurant or museum, for example, but otherwise, having ads constantly in your periphery could disrupt the privacy that some people look for in the ride-hailing experience.

Since one of the selling points of the Holograktor is that it learns your routes, habits, and preferences, and can even anticipate your next trip, the issue of data usage needs to be addressed. With the increasingly large catalog of misbehavior by internet companies, both regulators and consumers are beginning to have doubts about the way data is handled. If Holograktor offers the data it collects to tech companies and other advertisers, that could result in highly personalized and possibly invasive ads. With public trust in Big Tech companies at incredibly low levels, it is not clear if consumers would be willing to give up this data for a reduced ride price.

Nissan has also made a move into the metaverse with its I2V Invisible to Visible AR concept, which it launched at CES 2019. The system highlights obstacles that might not be obvious to the driver through a display inside the vehicle to enhance safety and driver comfort. If the driver is wearing AR goggles, the system also allows individuals to appear in the vehicle’s passenger seat as a 3D avatar. This might be a family member or friend to keep you company on a long drive, or a local guide to answer questions and give recommendations. Nissan hopes to roll out the technology in its vehicles from 2025.

Again, the new social and educational opportunities that could come from this are exciting. By merging the real and virtual worlds, individuals at home could join friends on trips anywhere around the world. Those usually long and boring drives can then be interjected with the avatars of your loved ones. Drivers could experience the world around them through the eyes of a local, or even in a completely different time period through AR overlays.

However, the main issue — and the more general barrier to widespread adoption of the metaverse and AR in vehicles — is that being able to see the outside world clearly is a safety-critical issue when a human is driving. If the AR overlay is not accurate enough, drivers could be misguided and end up in accidents. If the overlays are too distracting, drivers may miss important information about obstacles and obstructions. These issues must be of central importance to automotive companies looking to adopt AR in vehicles.

Emilio Campa is an Analyst on the Thematic Team at data analytics and consultancy GlobalData.