Comment For many, apps like ChatGPT, Copilot, Midjourney, or Gemini are generative AI.

But if there was one takeaway from Nvidia CEO Jensen Huang's GTC keynote, it's that, while ChatGPT is neat and it opened the world's eyes to large language models (LLMs), it only scratches the surface of the technology's potential - to sell GPUs that is.

While much of the fanfare went to Nvidia's new Blackwell chips, a very good proportion of Huang's two-hour presentation focused on the more tangible applications of AI whether they be for offices, manufacturing plants, warehouses, medical research, or robotics.

It's not hard to see why. The models that power ChatGPT and its contemporaries are massive, ranging from hundreds of billions to trillions of parameters. They're so large that training them often requires tens of thousands of GPUs running for weeks on end.

This, along with a desperate scramble by large enterprises to integrate AI into their operations, has fueled demand for accelerators. The major cloud providers and hyperscalers have been at the forefront of this buying up tens and even hundreds of thousands of GPUs for this purpose.

To be clear, these efforts have proven incredibly lucrative for Nvidia, which has seen its revenues more than double over the past fiscal year. Today, the company's market cap hovers at more than $2 trillion.

However, the number of companies that can afford to develop these models is relatively small. And making matters worse, many of the early attempts to commercialize the products of these efforts have proven lackluster, problematic, and generally unconvincing as to their value.

A recent report found that testers of Microsoft's Copilot services had a tough time justifying its $30/mo price tag despite many finding it useful.

Today, LLMs for things like chatbots and text-to-image generators are what's moving GPUs, but it's clear that Nvidia isn't putting all of its eggs in one basket. And, as usual, they aren't waiting around for others to create markets for its hardware.

Code? Where we're going we don't need code

One of the first places we might see this come to fruition is making it easier for smaller enterprises that don't have billion dollar R&D budgets to build AI accelerated apps.

We looked at this in more detail earlier this week, but the idea is that rather than training one big model to do a bunch of tasks, these AI apps will function a bit like an assembly line with multiple pre-trained or fine-tuned models responsible for various aspects of the job.

You can imagine using an app like this to automatically pull sales data, analyze it, and summarize the results in a neatly formatted report. Assuming the models can be trusted not to hallucinate data points, this approach should, at least in theory, lower the barrier to building AI apps.

Nvidia is doing this using NIMs, which are essentially just containerized models optimized for its particular flavor of infrastructure.

More importantly for Nvidia, the AI container runtime is part of its AI Enterprise suite, which will run you $4,500/year per GPU or $1/hour per GPU in the cloud. This means that even if Nvidia can't convince you to buy more GPUs, it can still extract annual revenues for the ones you already own or rent.

Warehouse tycoon 2

While stringing together a bunch of LLMs to generate reports is great and all, Huang remains convinced that AI also has applications in the physical world.

For the past few years, he's been pushing the idea of using its DGX and OVX systems to generate photo-realistic digital twins of factory floors, warehouses, and shipping operations, and this spring's GTC is no different.

According to Huang, these digital twins can simulate whether operational changes will bear fruit before they're implemented in the real world or help identify design flaws before construction even begins.

Huang's keynote was peppered with digital simulations which leads us to believe that he must have been a huge fan of RollerCoaster Tycoon or SimCity back in the day and thought: what if we do the same for everything.

But apparently, these virtual worlds can be quite useful at driving efficiencies and reducing operating costs. Nvidia claims that by using a digital twin to test and optimize factory floor layouts, Wistron - which produces its DGX servers - was able to boost worker efficiency by 51 percent, reduce cycle times by 50 percent, and curb defect rates by 40 percent.

While these digital twins may be able to help customers avoid costly mistakes, they're also an excuse for Nvidia to sell even more GPUs as the accelerators used in its OVX systems differ from the ones in its AI-centric DGX systems.

I am GR00T

Apparently, these digital twins are also useful for training robots to operate more independently on factory and warehouse floors.

Over the past few years, Nvidia has developed a variety of hardware and software platforms aimed at robotics. At GTC24, Huang revealed a new hardware platform called Jetson Thor alongside a foundation model called General Robotics 00 Technology, or GR00T for short, which are aimed at accelerating development of humanoid robots.

"In a way, human robotics is likely easier. The reason for that is because we have a lot more imitation training data that we can provide the robots because we're constructed in a very similar way," he explained.

How Nvidia plans to train these robots sounds to us a bit like how Neo learned kung fu in The Matrix. GR00T is trained using a dataset consisting of live and simulated video and other human imagery. The model is then further refined in a virtual environment that Nvidia calls Isaac Reinforcement Learning Gym. In this environment, a simulated robot running GR00T can learn to interact with the physical world.

This refined model can then be deployed to robots based on Nvidia's Jetson Thor compute platform.

Bigger models for bigger problems

While Nvidia's AI strategy isn't limited to training LLMs, Huang still believes bigger and more capable models will ultimately be necessary.

"We need even larger models. We're gonna train it with multimodality data, not just text on the internet. We're going to train it on texts and images and graphs and charts," he said. "And just as we learn watching TV, there's going to be a whole bunch of watching video, so that these models can be grounded in physics and understand that an arm doesn't go through a wall."

But of course the CEO of the world's largest supplier of AI infrastructure would say that. Nvidia is selling the shovels in this AI gold rush. And just like the crypto-crash that followed the Ethereum merge, Nvidia is, as always, looking ahead to its next big opportunity. ®

https://www.theregister.com//2024/03/25/nvidia_ai_bubble/