How generative AI could create assets for the metaverse | Jensen Huang

The metaverse skyrocketed into our collective awareness during the height of the pandemic, when people longed for better ways to connect with each other than video calls. Gaming’s hot growth during the pandemic also pushed it forward. But the metaverse became so trendy that it now faces a backlash, and folks aren’t talking about it as much.

Yet technologies that will power the metaverse are speeding ahead. One of those technologies is generative AI, which uses deep learning neural networks to produce creative concept art and other ideas based on simple text prompts.

Jensen Huang, CEO of AI and graphics chip maker Nvidia, believes that generative AI will be transformational and it’s just getting started. One of its biggest applications could be with the metaverse, which has huge demands for content as developers need to fill out virtual worlds with 3D assets. And numerous companies like Stable Diffusion, Promethean AI and Ludo AI are using these technologies to automatically generate artwork and other assets for gaming and metaverse applications. Nvidia has its own research going on this front.

Many metaverse companies are hoping that generative AI will help provide the resources to help them build out their worlds. Huang believes you will see progress when you enter more and more prompts — such as text to flesh out a concept — and the concept imagery gets better and better. And he also believes that when it becomes reusable across different Omniverse applications, then it will be clear that generative AI has reached a more mature stage.

I recently caught up with Huang for a short interview on the metaverse and gaming. Our GamesBeat Summit: Into the Metaverse 3 event is coming on February 1 to February 2.

Here’s an edited transcript of our interview.

GamesBeat: I wonder if there are some people losing their enthusiasm for the metaverse now. It was last year’s news. Maybe they think it’s an idea that might die during a downturn. What would you say about that?

Jensen Huang: First, the definition of “metaverse,” as it pertains to the virtual environment where we develop artificial intelligence software. I believe Omniverse, metaverse, is essential to that journey. You cannot develop software you can’t test. You can’t deploy software you can’t test. With the world wanting to become software-defined, where there’s a software-defined car or software-defined lawn mower or software-defined farming equipment or software-defined pick-and-place or a software-defined warehouse, we can’t move to that new world without a virtual environment to simulate and test that software. That’s what Omniverse is built for in the first place. It’s sensible to me that the next era of AI will require Omniverse, or what some people call the industrial metaverse.

The day when it comes to commercial product — for example, marketing home appliances — the ability to have a digital version of that product that you can virtually place in the environment you’re in could be very helpful to shopping. I believe that in the long term, the idea of looking at a 2D image for shopping, digital shopping, versus seeing a 3D object in its environment — I think it’s very clear that metaverse will be used for commerce. Whether the metaverse will be used for video conferencing — which I think it will be — and other applications, it’s just a matter of time. The examples that I’ve expressed absolutely will happen.

GamesBeat: In this era of generative AI, how do you think that’s going to help the metaverse to happen?

Huang: Without generative AI, how would consumers be able to create virtual worlds without great effort? Now you can create virtual worlds in 3D, which is coming around the corner. You can see it about to happen. We can now do generative AI for images. We can do it for videos. At the rate that it’s moving, you’ll do it for entire villages; 3D villages and landscapes and cities and so on. You’ll be able to assemble an example of an image and generate an entire 3D world. That’s going to enable the metaverse like you can’t believe. You need it. Absolutely.

That’s exactly the reason why I always say, when we’re talking about the metaverse, when we’re talking about Omniverse, that AI and Omniverse go hand in hand. To me it’s the same thing.

GamesBeat: A forecast from Newzoo came out and it said gaming may shrink as an industry for the first time in more than a decade, year over year. What do you think about that? Is that a concern, or do you see that as something that would be a correction before we resume growth in gaming?

Huang: In fact, if you go back, you’ll notice that PC gaming has been slow relative to the other platforms for some time. The reason for that is because PC gaming is more than just what’s measurable through packaged game sales or digital download sales.

Where gaming starts and where creativity starts is really hard to tell anymore. So many people are influencers and video bloggers and avatars. The definition of gaming is a lot broader on PC. If you look at one of the most important applications on the PC at the moment, it’s generative AI. All these people who are creating images of animation and images of games. Gaming for PCs has always been an expanding category that’s reinventing itself and redefining itself.