How deep learning will ignite the metaverse in 2023 and beyond

The metaverse is becoming one of the hottest topics not only in technology but in the social and economic spheres. Tech giants and startups alike are already working on creating services for this new digital reality.

The metaverse is slowly evolving into a mainstream virtual world where you can work, learn, shop, be entertained and interact with others in ways never before possible. Gartner recently listed the metaverse as one of the top strategic technology trends for 2023, and predicts that by 2026, 25% of the population will spend at least one hour a day there for work, shopping, education, social activities and/or entertainment. That means organizations that use the metaverse effectively will be able to engage with both human and machine customers and create new revenue streams and markets.

However, most of these metaverse experiences will be able to continue to progress only with the use of deep learning (DL), as artificial intelligence (AI) and data science will be at the forefront of advancing this technology. For example, deep learning algorithms are making computers better at gesture recognition and eye tracking, thanks to the latest developments in computer vision that enable natural interactions and better understanding of emotion and body language. As such technologies are an essential aspect of the metaverse’s immersive interface, deep learning technologies now aim to further enhance realistic AI storytelling, creative partnering and machine understanding. 

Shaping a new reality with deep learning  

Currently, the digital realities being developed by different companies have their own attributes and integrated functionalities, and are at different development levels. Many of these multiverse platforms are expected to converge, and this junction is where AI and data science domains, such as deep learning, will be critical in taking users to a new stage in their metaverse journey. Success in these endeavors will be contingent upon understanding vital elements of the algorithmic models and their metrics.

Deep learning-based software is already being integrated into virtual worlds; some examples include autonomously driving chatbots and other forms of natural language processing to ensure seamless interactions. For another example, in AR technology, deep learning-enabled AI is used in camera pose estimation, immersive rendering, real-world object detection and 3D object reconstruction, helping to guarantee the variety and usability of AR applications.

In October, Meta announced the launch of its universal speech translator (UST) project, which aims to create AI systems that enable real-time speech-to-speech translation across all languages — regardless of the user’s language. In addition, the company’s recent advances in unsupervised speech recognition (wav2vec-U) and unsupervised machine translation (mBART) will aid the future work of translating more spoken languages within the metaverse. 

All such implementations require massive training data and modeling, now made possible through deep learning methodologies. In addition, AI-based Web3 technologies are now being called upon to automate smart contracts and decentralized ledgers, and create universal blockchain technologies to enable virtual transactions. 

“Deep learning provides much higher accuracy [and] almost no false positives, and if properly implemented, eliminates data noise (corruption),” Jerrod Piker, competitive intelligence analyst at Deep Instinct, told VentureBeat. 

New ways to interact

Piker said that such implementations could aid in improving the metaverse, as a deep learning model is trained on all available data, providing incredible results on image recognition and natural language processing. 

“Meta has applied this in translating code from one programming language to another. Since the metaverse is a wide and open world, automatically translating code can have a huge impact on seamless integration between different platforms within the metaverse,” he said. 

Likewise, Scott Stephenson, CEO and cofounder at Deepgram, believes that deep neural networks are more capable and sophisticated than neural networks with fewer layers.

“Companies have an interesting opportunity for their customers and community to interact with their brand(s) in new and exciting ways, and deep learning-based artificial intelligence plays a major role in facilitating those experiences,” said Stephenson. 

He explained that companies can now have AI brand representatives — trained on a company’s unique linguistic style and product documentation — wander about the metaverse, evangelizing whatever product or service the company seeks to promote. 

“Rather than giving them dozens or even hundreds of lines of pre-scripted dialogue (like what you’d experience in most video games these days), there’s no reason why a metaverse platform shouldn’t be running a generative text chatbot in the background to drive conversation and engagement,” he said.

Overcoming challenges through adoption 

Despite its promise and potential, the metaverse continues to face user-based risks, such as data security. Deep learning-based AI models could be instrumental in overcoming those challenges when integrated with legacy tools.

“Securing sensitive data that is being created, sent and shared across the metaverse requires more advanced techniques than past data security efforts. Deep learning can provide excellent results on this front with its uncanny ability to accurately identify content,” said Piker. “For instance, ongoing inspection for certain sensitive data to ensure it is not being leaked outside of its intended channel is extremely important, and deep learning is unmatched in correctly and efficiently identifying digital content of all kinds, with a far superior false positive rate vs. other machine learning models.” 

Scott Likens, innovation and trust technology leader at PwC, said that many brands have started to see the metaverse’s actual business value as deep learning and AI converge with VR to provide a much deeper experience for the metaverse in the future. 

“The generation of assets in the metaverse now benefits from AI, as there is currently a lack of content and digital assets to fill the metaverse. In addition, with the advances in data collection through IoT, we can feed the data-hungry deep learning models to create lifelike yet synthetic worlds that are being used to help drive business strategy and more at a pace we can’t match in the current workforce,” said Likens. 

“Deep learning technologies are going to be highly important in terms of automation,” says Patrik Wilkens, vice president of operations at TheSoul Publishing, whose universe of well-known channels includes 5-Minute Crafts, Bright Side and 123 GO!. 

“Progress that used to take hours and hours of human effort is now attainable with incredible efficiency. As tech companies and content creators utilize the best tech, incorporating deep learning into their processes, the manpower that was previously used to make things work can now be used on other things. This is especially important for creative domains,” Wilkens told VentureBeat. 

Wilkens further explained that his company, TheSoul, is currently utilizing deep learning-based algorithms for several metaverse use cases. 

“We are using deep learning-based artificial intelligence in our content right now to proofread, translate, [perform] quality assurance (QA) and build graphics. We’re also in the development stage on a number of initiatives, including the 5-Minute Crafts marketplace within the metaverse,” he said. “That could work by your avatar walking into TheSoul’s shopping mall-style building, watching a craft video, and going to the AI assistant to help you purchase the materials needed to complete the project.”

What to expect from the metaverse in 2023

Adrian McDermott, CTO at Zendesk, believes that in 2023, we can expect to see deep learning and AI technologies power and scale customer self-service in the metaverse. 

“Businesses will expand the use of AI and automation to route and escalate urgent user issues in real time, ensuring the experience remains seamless,” McDermott told VentureBeat. “Large language models (LLMs) will play a role in helping brands understand customer needs in these new spaces, as well as generating potential responses to service requests. Self-service powered by deep learning-based AI models can unburden human agents by helping customers sort through straightforward questions more easily, freeing agents up to dig into the more difficult cases.”

McDermott said that we would begin to see industries beyond retail and gaming begin to build or pilot metaverse experiences to stay competitive. Brands will be using the metaverse to not only engage with customers, but to build loyalty through digital collectibles, and automation will play a role in that journey. 

“Don’t be surprised to see not only an expansion of digital storefront and concert experiences, but also increased use by the enterprise for hosting meetings, training and upskilling employees on critical job skills,” he said.   

Likewise, Wilkens predicts that in 2023, we can expect brands to begin building communities around virtual influencers. 

“Brands will focus on developing more ‘meaningful’ content to engage their virtual influencers’ communities in an effort to be more human and connect with audiences authentically,” he said. ”Additionally, we expect to see a rise of avatars. They will be everywhere — especially in the metaverse — and will evolve dramatically on platforms like Snapchat due to new features like avatar fashion and digital items coming in 2023.”