Naveen Rao, a neuroscientist turned tech entrepreneur, once tried to compete with Nvidia, the world’s leading maker of chips designed for artificial intelligence.
At a startup that was later bought by semiconductor giant Intel, Mr. Rao worked on chips intended to replace Nvidia’s GPUs, components adapted for artificial intelligence tasks such as machine learning. Mr. Rao said that while Intel moved slowly, Nvidia quickly upgraded its products with new AI features that countered what it was developing.
After leaving Intel and leading software startup MosaicML, Mr. Rao took Nvidia chips and benchmarked them against those from competitors. He found that Nvidia has distinguished itself outside of chips by creating a large community of AI programmers who are constantly inventing using the company’s technology.
“Everyone builds on Nvidia first,” said Mr. Rao. “If you come out with a new piece of hardware, you’re racing to catch up.”
For more than 10 years, Nvidia has built a nearly impregnable lead in producing chips that can perform complex AI tasks like image, facial, and speech recognition, as well as generate text for chatbots like ChatGPT. This once nascent industry achieved this dominance by recognizing the AI trend early on, dedicating its chips to those tasks and then developing key pieces of software that help advance AI.
Since then, Jensen Huang, co-founder and CEO of Nvidia, has continued to raise the bar. To maintain its leading position, his company offered clients access to specialized computers, computing services, and other tools of their emerging trade. This has turned Nvidia, for all intents and purposes, into a one-stop-shop for AI development.
While Google, Amazon, Meta, IBM and others have also produced AI chips, today Nvidia accounts for more than 70 percent of AI chip sales and holds a larger position in training generative AI models, according to research firm Omdia.
In May, the company’s status as the clear winner in the AI revolution became clear when it projected a 64 percent jump in quarterly revenue, much more than Wall Street had predicted. On Wednesday, Nvidia — which has surpassed $1 trillion in market capitalization to become the world’s most valuable chip maker — is expected to confirm those record results and provide more signals about booming demand for artificial intelligence.
“Customers will wait 18 months to buy an Nvidia system rather than buy an available off-the-shelf chip from another startup or competitor,” said Daniel Neumann, an analyst at Futurum Group. “It’s unbelievable.”
Mr Huang, 60, known for his trademark black leather jacket, talked about AI for years before becoming one of the most popular faces in the movement. He has said publicly that computing is undergoing its biggest transformation since IBM defined how most systems and software work 60 years ago. Now, he said, GPUs and other special-purpose chips are replacing standard microprocessors, and AI chatbots are replacing complex software coding.
“The thing we understood is that this is a reinvention of how computing works,” Mr. Huang said in an interview. “And we built everything from the ground up, from the processor all the way to the end.”
Mr. Huang helped start Nvidia in 1993 to make chips that display images in video games. While standard microprocessors excel at performing complex calculations sequentially, the company’s GPUs perform many simple tasks simultaneously.
In 2006, Mr. Huang took this further. He announced a software technology called CUDA that helped program graphics processing units to perform new tasks, turning them from single-purpose chips into general-purpose chips that could take over other functions in areas such as physics and chemical simulation.
A major breakthrough came in 2012, when researchers used GPUs to achieve human-like accuracy on tasks such as recognizing a cat in a photo—a precursor to recent developments such as creating images from text prompts.
Nvidia responded by transforming “every aspect of our company to develop this new field,” Mr. Jensen said recently in his opening speech at National Taiwan University.
The effort, which the company estimates has cost more than $30 billion over a decade, has made Nvidia more than just a component supplier. Besides collaborating with leading scientists and startups, the company has built a team that is directly involved in AI activities such as creating and training language models.
Forewarning about what AI practitioners need has led Nvidia to develop several major software layers beyond CUDA. Those included hundreds of pre-built pieces of code called libraries which saved programmers labor.
In hardware, Nvidia has earned a reputation for consistently delivering faster chips every two years. In 2017, it started modifying GPUs to handle specific AI computations.
In the same year, Nvidia, which usually sells chips or circuit boards for other companies’ systems, began selling entire computers to carry out artificial intelligence tasks more efficiently. Some of its systems are now the size of supercomputers, which it assembles and runs using proprietary networking technology and thousands of GPUs. It can take weeks for these machines to train the latest AI models.
“This kind of computing doesn’t allow you to just build a chip and customers use it,” Mr. Huang said in the interview. “You have to build the entire data center.”
Last September, Nvidia announced new chips called the H100, which it beefed up to handle so-called Switch operations. Such computations have turned out to be the basis for services like ChatGPT, which have driven what Mr Huang calls an “iPhone moment” of generative AI.
To expand its influence further, Nvidia has also partnered with major technology companies and invested in high-profile AI startups that use its chips. One of them was Inflection AI, which in June announced $1.3 billion in funding from Nvidia and others. The money was used to help finance the purchase of 22,000 H100 chips.
Mostafa Soliman, CEO of Inflection, said there was no obligation to use Nvidia products but that competitors had not offered a viable alternative. He said, “None of them come close.”
Nvidia has also been funneling cash and its rare H100s recently into cloud services like CoreWeave, which allow companies to rent out time on computers instead of buying their own. CoreWeave, which will operate Inflection hardware and owns more than 45,000 Nvidia chips, raised $2.3 billion in debt this month to help buy more.
Given the demand for its chips, Nvidia must decide who gets how many of its chips. This power makes some tech executives uncomfortable.
“It’s really important that hardware not become a bottleneck for AI or a gatekeeper for AI,” said Clement Delange, CEO of Hugging Face, an online repository of language models that collaborates with Nvidia and its competitors.
Some competitors said it was difficult to compete with a company that sells computers, software, cloud services, and trained AI models, as well as processors.
“Unlike any other chip company, they were willing to openly compete with their customers,” said Andrew Feldman, CEO of Cerebras, a startup developing AI chips.
But few customers complain, at least in public. Even Google, which began creating competing AI chips more than a decade ago, relies on Nvidia’s GPUs for some of its work.
The demand for Google chips is “enormous,” said Amin Vahdat, Google’s vice president and general manager of computing infrastructure. But he added, “We work very closely with Nvidia.”
Nvidia doesn’t discuss pricing or chip allocation policies, but industry executives and analysts have said each H100 costs between $15,000 to more than $40,000, depending on packaging and other factors — roughly two to three times more than the previous A100 chip.
Pricing “is one of the places where Nvidia has left a lot of room for other people to compete,” said David Brown, vice president of Amazon’s cloud unit, arguing that its AI chips are a bargain compared to the Nvidia chips it also uses.
Mr. Huang said that the outstanding performance of its chipset saves customers money. “If you can cut the training time in half in a $5 billion data center, the savings are more than the cost of all the chips,” he said. “We are the lowest cost solution in the world.”
He’s also begun touting a new product, the Grace Hopper, that combines internally developed graphics processing units and microprocessors, to counter chips that competitors say use much less power to run AI services.
However, more competition seems inevitable. One of the most promising in the race is the graphics processing unit (GPU) sold by Advanced Micro Devices, said Mr. Rao, whose startup recently bought data and artificial intelligence company DataBricks.
“No matter how anyone wants to say it’s all done, it’s not all over,” AMD CEO Lisa Su said.
Kid Metz Contribute to the preparation of reports.