Nvidia dominates the AI chip market, but there’s more competition than ever
Jensen Huang, co-founder and chief executive officer of Nvidia Corp., during the Nvidia GPU Technology Conference (GTC) in San Jose, California, US, on Tuesday, March 19, 2024.
David Paul Morris | Bloomberg | Getty Images
Nvidia’s 27% rally in May pushed its market cap to $2.7 trillion, behind only Microsoft and Apple among the most-valuable public companies in the world. The chipmaker reported a tripling in year-over-year sales for the third straight quarter driven by soaring demand for its artificial intelligence processors.
Mizuho Securities estimates that Nvidia controls between 70% and 95% of the market for AI chips used for training and deploying models like OpenAI’s GPT. Underscoring Nvidia’s pricing power is a 78% gross margin, a stunningly high number for a hardware company that has to manufacture and ship physical products.
Rival chipmakers Intel and Advanced Micro Devices reported gross margins in the latest quarter of 41% and 47%, respectively.
Nvidia’s position in the AI chip market has been described as a moat by some experts. Its flagship AI graphics processing units (GPUs), such as the H100, coupled with the company’s CUDA software led to such a head start on the competition that switching to an alternative can seem almost unthinkable.
Still, Nvidia CEO Jensen Huang, whose net worth has swelled from $3 billion to about $90 billion in the past five years, has said he’s “worried and concerned” about his 31-year-old company losing its edge. He acknowledged at a conference late last year that there are many powerful competitors on the rise.
“I don’t think people are trying to put me out of business,” Huang said in November. “I probably know they’re trying to, so that’s different.”
Nvidia has committed to releasing a new AI chip architecture every year, rather than every other year as was the case historically, and to putting out new software that could more deeply entrench its chips in AI software.
But Nvidia’s GPU isn’t alone in being able to run the complex math that underpins generative AI. If less powerful chips can do the same work, Huang might be justifiably paranoid.
The transition from training AI models to what’s called inference — or deploying the models — could also give companies an opportunity to replace Nvidia’s GPUs, especially if they’re less expensive to buy and run. Nvidia’s flagship chip costs roughly $30,000 or more, giving customers plenty of incentive to seek alternatives.
“Nvidia would love to have 100% of it, but customers would not love for Nvidia to have 100% of it,” said Sid Sheth, co-founder of aspiring rival D-Matrix. “It’s just too big of an opportunity. It would be too unhealthy if any one company took all of it.”
Founded in 2019, D-Matrix plans to release a semiconductor card for servers later this year that aims to reduce the cost and latency of running AI models. The company raised $110 million in September.
In addition to D-Matrix, companies ranging from multinational corporations to nascent startups are fighting for a slice of the AI chip market that could reach $400 billion in annual sales in the next five years, according to market analysts and AMD. Nvidia has generated about $80 billion in revenue over the past four quarters, and Bank of America estimates the company sold $34.5 billion in AI chips last year.
Many companies taking on Nvidia’s GPUs are betting that a different architecture or certain trade-offs could produce a better chip for particular tasks. Device makers are also developing technology that could end up doing a lot of the computing for AI that’s currently taking place in large GPU-based clusters in the cloud.
“Nobody can deny that today Nvidia is the hardware you want to train and run AI models,” Fernando Vidal, co-founder of 3Fourteen Research, told CNBC. “But there’s been incremental progress in leveling the playing field, from hyperscalers working on their own chips, to even little startups, designing their own silicon.”
AMD CEO Lisa Su wants investors to believe there’s plenty of room for many successful companies in the space.
“The key is that there are a lot of options there,” Su told reporters in December, when her company launched its most recent AI chip. “I think we’re going to see a situation where there’s not only one solution, there will be multiple solutions.”
Other big chipmakers
Lisa Su displays an AMD Instinct MI300 chip as she delivers a keynote address at CES 2023 in Las Vegas, Nevada, on Jan. 4, 2023.
David Becker | Getty Images
AMD makes GPUs for gaming and, like Nvidia, is adapting them for AI inside of data centers. Its flagship chip is the Instinct MI300X. Microsoft has already bought AMD processors, offering access to them through its Azure cloud.
At launch, Su highlighted the chip’s excellence at inference, as opposed to competing with Nvidia for training. Last week, Microsoft said it was using AMD Instinct GPUs to serve its Copilot models. Morgan Stanley analysts took the news as a sign that AMD’s AI chip sales could surpass $4 billion this year, the company’s public target.
Intel, which was surpassed by Nvidia last year in terms of revenue, is also trying to establish a presence in AI. The company recently announced the third version of its AI accelerator, Gaudi 3. This time Intel compared it directly to the competition, describing it as a more cost-effective alternative and better than Nvidia’s H100 in terms of running inference, while faster at training models.
Bank of America analysts estimated recently that Intel will have less than 1% of the AI chip market this year. Intel says it has a $2 billion order of backlogs for the chip.
The main roadblock to broader adoption may be software. AMD and Intel are both participating in a big industry group called the UXL foundation, which includes Google, that’s working to create free alternatives to Nvidia’s CUDA for controlling hardware for AI applications.
Nvidia’s top customers
One potential challenge for Nvidia is that it’s competing against some of its biggest customers. Cloud providers including Google, Microsoft and Amazon are all building processors for internal use. The Big Tech three, plus Oracle, make up over 40% of Nvidia’s revenue.
Amazon introduced its own AI-oriented chips in 2018, under the Inferentia brand name. Inferentia is now on its second version. In 2021, Amazon Web Services debuted Tranium targeted to training. Customers can’t buy the chips but they can rent systems through AWS, which markets the chips as more cost efficient than Nvidia’s.
Google is perhaps the cloud provider most committed to its own silicon. The company has been using what it calls Tensor Processing Units (TPUs) since 2015 to train and deploy AI models. In May, Google announced the sixth version of its chip, Trillium, which the company said was used to develop its models, including Gemini and Imagen.
Google also uses Nvidia chips and offers them through its cloud.
Microsoft isn’t as far along. The company said last year that it was building its own AI accelerator and processor, called Maia and Cobalt.
Meta isn’t a cloud provider, but the company needs massive amounts of computing power to run its software and website and to serve ads. While the Facebook parent company is buying billions of dollars worth of Nvidia processors, it said in April that some of its homegrown chips were already in data centers and enabled “greater efficiency” compared to GPUs.
JPMorgan analysts estimated in May that the market for building custom chips for big cloud providers could be worth as much as $30 billion, with potential growth of 20% per year.
Startups
Cerebras’ WSE-3 chip is one example of new silicon from upstarts designed to run and train artificial intelligence.
Cerebras Systems
Venture capitalists see opportunities for emerging companies to jump into the game. They invested $6 billion in AI semiconductor companies in 2023, up slightly from $5.7 billion a year earlier, according to data from PitchBook.
It’s a tough area for startups as semiconductors are expensive to design, develop and manufacture. But there are opportunities for differentiation.
For Cerebras Systems, an AI chipmaker in Silicon Valley, the focus is on basic operations and bottlenecks for AI, versus the more general purpose nature of a GPU. The company was founded in 2015 and was valued at $4 billion during its most recent fundraising, according to Bloomberg.
The Cerebras chip, WSE-2, puts GPU capabilities as well as central processing and additional memory into a single device, which is better for training large models, said CEO Andrew Feldman.
“We use a giant chip, they use a lot of little chips,” Feldman said. “They’ve got challenges of moving data around, we don’t.”
Feldman said his company, which counts Mayo Clinic, GlaxoSmithKline, and the U.S. Military as clients, is winning business for its supercomputing systems even going up against Nvidia.
“There’s ample competition and I think that’s healthy for the ecosystem,” Feldman said.
Sheth from D-Matrix said his company plans to release a card with its chiplet later this year that will allow for more computation in memory, as opposed to on a chip like a GPU. D-Matrix’s product can be slotted into an AI server along existing GPUs, but it takes work off of Nvidia chips, and helps to lower the cost of generative AI.
Customers “are very receptive and very incentivized to enable a new solution to come to market,” Sheth said.
Apple and Qualcomm
Apple iPhone 15 series devices are displayed for sale at The Grove Apple retail store on release day in Los Angeles, California, on September 22, 2023.
Patrick T. Fallon | Afp | Getty Images
The biggest threat to Nvidia’s data center business may be a change in where processing happens.
Developers are increasingly betting that AI work will move from server farms to the laptops, PCs and phones we own.
Big models like the ones developed by OpenAI require massive clusters of powerful GPUs for inference, but companies like Apple and Microsoft are developing “small models” that require less power and data and can run on a battery-powered device. They may not be as skilled as the latest version of ChatGPT, but there are other applications they perform, such as summarizing text or visual search.
Apple and Qualcomm are updating their chips to run AI more efficiently, adding specialized sections for AI models called neural processors, which can have privacy and speed advantages.
Qualcomm recently announced a PC chip that will allow laptops to run Microsoft AI services on the device. The company has also invested in a number of chipmakers making lower-power processors to run AI algorithms outside of a smartphone or laptop.
Apple has been marketing its latest laptops and tablets as optimized for AI because of the neural engine on its chips. At its upcoming developer conference, Apple is planning to show off a slew of new AI features, likely running on the company’s iPhone-powering silicon.
Jaque Silva | Nurphoto | Getty Images
Oracle’s Federal Electronic Health Record experienced a nation-wide outage on Tuesday, the Department of Veterans Affairs confirmed to CNBC.
The agency said “all users” of the company’s Federal EHR, including the VA, the Department of Defense, the U.S. Coast Guard and the National Oceanic and Atmospheric Administration, were impacted. Six VA medical centers, 26 community clinics, and remote VA sites experienced disruptions, the agency said.
“Affected VA medical facilities followed standard contingency procedures during the outage to ensure continuity of care for Veterans,” a VA spokesperson said in a statement Thursday.
An electronic health record, or an EHR, is a digital version of a patient’s medical history that’s updated by doctors and nurses. It’s crucial software within the U.S. health-care system, and outages can cause serious disruptions to patient care.
Oracle is one of the largest EHR vendors thanks to it’s $28 billion acquisition of the medical records giant Cerner in 2022.
The company’s Federal EHR initially started experiencing issues at around 8:37 a.m. Eastern on Tuesday, the VA said. Users reported that the software froze and they were unable to access applications. Access was restored and cleared by 2:05 p.m. Eastern that day after Oracle restarted the system.
Oracle is carrying out an investigation to determine what caused the outage, the VA said. Oracle did not immediately respond to CNBC’s request for comment.
The outage marks Oracle’s latest stumble in a thorny, years-long EHR rollout with the VA, which has been marred by patient safety concerns. The agency launched a strategic review of Cerner in 2021, before Oracle’s acquisition, and it temporarily paused deployment of the software in 2023.
Four VA facilities in Michigan are slated to deploy Oracle’s Federal EHR in 2026.
In October, Oracle unveiled a brand-new EHR equipped with fresh cloud and artificial intelligence capabilities. The early adopter program for the software begins this year, though it’s not clear if the VA has plans to utilize it.
Oracle is slated to report third-quarter fiscal 2025 earnings on Monday.
Watch: Oracle CEO Safra Catz: Being number one is very important
Broadcom CEO Hock Tan.
Lucas Jackson | Reuters
Broadcom reported first-quarter earnings on Thursday that topped analysts’ expectations, and the chipmaker offered strong guidance for the current quarter. The stock jumped 16% in extended trading.
Here’s how the company did versus LSEG consensus estimates:
- Earnings per share: $1.60 adjusted vs. $1.49 expected
- Revenue: $14.92 billion vs. $14.61 billion expected
Broadcom said it expects about $14.9 billion in second-quarter revenue, higher than the $14.76 billion forecast by Wall Street analysts. Revenue in the last quarter rose 25% from $11.96 billion a year earlier.
The company said net income increased to $5.5 billion, or $1.14 per share, from $1.33 billion, or 28 cents per share, in the same period last year.
Broadcom’s artificial intelligence business is at the center of the company’s recent boom, which saw its stock price more than double last year. The company is one of the primary data center infrastructure vendors for AI, working both on Google’s custom AI chips as well as providing essential components for networking thousands of other chips together to develop advanced AI software.
Prior to the after-hours pop, the stock was down about 23% so far in 2025, as investors rotate out of risk partly due to concern about President Donald Trump’s tariffs.
Broadcom said it recorded $4.1 billion in AI revenue during the first quarter, which is 77% higher on a year-over-year basis. Those sales are reported as part of Broadcom’s semiconductor solutions business, which grew 11% on an annual basis to $8.21 billion during the quarter.
Broadcom CEO Hock Tan said in a statement that the company expects “continued strength in AI semiconductor revenue,” reaching a projected $4.4 billion in the second quarter.
In December, Broadcom said it was developing custom AI chips with three large cloud customers. Tan said on Thursday that in addition to those customers, it had “deeply engaged” with two other hyperscalers, and are working with four other potential customers to develop their own custom AI chips.
Tan said that Broadcom closely chooses partners for developing custom AI chips who can deploy the resulting product in large quantities. “To put it bluntly, we don’t do it for startups,” Tan said.
The other major part of Broadcom’s revenue comes from its infrastructure software division, which includes software from the company’s acquisition of VMware in the fourth fiscal quarter of 2023. Broadcom said it saw $6.7 billion in software sales during the quarter, a 47% increase on an annual basis.
WATCH: Chip stocks see strong performance punished by markets
Antonio Neri, CEO of Hewlett Packard Enterprise, speaks during an interview with CNBC on the floor of the New York Stock Exchange (NYSE) in New York City, October 20, 2023.
Brendan McDermid | Reuters
Hewlett Packard Enterprise shares slid 19% in extended trading on Thursday as the data center equipment maker issued quarterly and full-year guidance that came in below consensus.
Here’s how the company did in the fiscal first quarter in comparison with LSEG consensus:
- Earnings per share: 49 cents adjusted vs. 49 cents expected
- Revenue: $7.85 billion vs. $7.82 billion expected
HPE’s revenue rose 16% year over year in the quarter ending on Jan. 31, according to a statement. The company was left with profit of $598 million, or 44 cents per share, up from $387 million, or 29 cents per share, in the same quarter a year earlier. The adjusted earnings per share excludes stock-based compensation.
“We could have executed better,” CEO Antonio Neri said on a conference call with analysts. The company had higher than normal inventory for artificial intelligence servers because of a shift to next-generation Blackwell graphics processing units from Nvidia.
The backlog for AI systems rose 29% quarter over quarter to $3.1 billion. Total server revenue totaled $4.29 billion.
HPE dealt with extensive discounting in the market while selling traditional servers during the quarter, finance chief Marie Myers said. As the quarter progressed, HPE moved to limit travel and discretionary spending, she said.
“We expect pricing adjustments may negatively impact top-line growth in the near term,” Myers said.
The company said it would implement a cost-cutting program involving layoffs over the next 18 months that will lead to $350 million in gross savings by the 2027 fiscal year. Around 2,500 employees will be affected, a spokesperson said, representing about 5% of the workforce when also factoring in expected attrition. At the end of October, HPE employed 61,000 people, according to its most recent annual report.
In January, the U.S. Justice Department filed in a federal district court to stop HPE from acquiring Juniper Networks. HPE announced the proposed $14 billion deal in January 2024. The court expects a trial to begin in July, according to the statement. The deal should close by October 2025, HPE said. In December, the company had said the transaction would be done in early 2025.
HPE called for 28 cents to 34 cents in adjusted earnings per share for the fiscal second quarter, with revenue coming in between $7.2 billion and $7.6 billion. Analysts surveyed by LSEG had looked for 50 cents per share on $7.93 billion in revenue.
For the 2025 fiscal year, HPE sees $1.70 to $1.90 in adjusted earnings per share. Analysts polled by LSEG had predicted $2.13 per share.
HPE expects to update its prices to reflect higher expenses from U.S. tariffs, Neri said, adding that he has not perceived any business deterioration from President Donald Trump’s so-called Department of Government Efficiency.
As of Thursday’s close, HPE shares were up about 2% so far in 2025, while the S&P 500 index was down 2%.
WATCH: HPE shares fall more than 10% after mixed earnings, layoff plans
-
Sports2 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports11 months agoStory injured on diving stop, exits Red Sox game
-
Sports1 year agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports3 years ago
Team Europe easily wins 4th straight Laver Cup
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike
-
Business2 years agoBank of England’s extraordinary response to government policy is almost unthinkable | Ed Conway