Nvidia dominates the AI chip market, but there’s more competition than ever
Jensen Huang, co-founder and chief executive officer of Nvidia Corp., during the Nvidia GPU Technology Conference (GTC) in San Jose, California, US, on Tuesday, March 19, 2024.
David Paul Morris | Bloomberg | Getty Images
Nvidia’s 27% rally in May pushed its market cap to $2.7 trillion, behind only Microsoft and Apple among the most-valuable public companies in the world. The chipmaker reported a tripling in year-over-year sales for the third straight quarter driven by soaring demand for its artificial intelligence processors.
Mizuho Securities estimates that Nvidia controls between 70% and 95% of the market for AI chips used for training and deploying models like OpenAI’s GPT. Underscoring Nvidia’s pricing power is a 78% gross margin, a stunningly high number for a hardware company that has to manufacture and ship physical products.
Rival chipmakers Intel and Advanced Micro Devices reported gross margins in the latest quarter of 41% and 47%, respectively.
Nvidia’s position in the AI chip market has been described as a moat by some experts. Its flagship AI graphics processing units (GPUs), such as the H100, coupled with the company’s CUDA software led to such a head start on the competition that switching to an alternative can seem almost unthinkable.
Still, Nvidia CEO Jensen Huang, whose net worth has swelled from $3 billion to about $90 billion in the past five years, has said he’s “worried and concerned” about his 31-year-old company losing its edge. He acknowledged at a conference late last year that there are many powerful competitors on the rise.
“I don’t think people are trying to put me out of business,” Huang said in November. “I probably know they’re trying to, so that’s different.”
Nvidia has committed to releasing a new AI chip architecture every year, rather than every other year as was the case historically, and to putting out new software that could more deeply entrench its chips in AI software.
But Nvidia’s GPU isn’t alone in being able to run the complex math that underpins generative AI. If less powerful chips can do the same work, Huang might be justifiably paranoid.
The transition from training AI models to what’s called inference — or deploying the models — could also give companies an opportunity to replace Nvidia’s GPUs, especially if they’re less expensive to buy and run. Nvidia’s flagship chip costs roughly $30,000 or more, giving customers plenty of incentive to seek alternatives.
“Nvidia would love to have 100% of it, but customers would not love for Nvidia to have 100% of it,” said Sid Sheth, co-founder of aspiring rival D-Matrix. “It’s just too big of an opportunity. It would be too unhealthy if any one company took all of it.”
Founded in 2019, D-Matrix plans to release a semiconductor card for servers later this year that aims to reduce the cost and latency of running AI models. The company raised $110 million in September.
In addition to D-Matrix, companies ranging from multinational corporations to nascent startups are fighting for a slice of the AI chip market that could reach $400 billion in annual sales in the next five years, according to market analysts and AMD. Nvidia has generated about $80 billion in revenue over the past four quarters, and Bank of America estimates the company sold $34.5 billion in AI chips last year.
Many companies taking on Nvidia’s GPUs are betting that a different architecture or certain trade-offs could produce a better chip for particular tasks. Device makers are also developing technology that could end up doing a lot of the computing for AI that’s currently taking place in large GPU-based clusters in the cloud.
“Nobody can deny that today Nvidia is the hardware you want to train and run AI models,” Fernando Vidal, co-founder of 3Fourteen Research, told CNBC. “But there’s been incremental progress in leveling the playing field, from hyperscalers working on their own chips, to even little startups, designing their own silicon.”
AMD CEO Lisa Su wants investors to believe there’s plenty of room for many successful companies in the space.
“The key is that there are a lot of options there,” Su told reporters in December, when her company launched its most recent AI chip. “I think we’re going to see a situation where there’s not only one solution, there will be multiple solutions.”
Other big chipmakers
Lisa Su displays an AMD Instinct MI300 chip as she delivers a keynote address at CES 2023 in Las Vegas, Nevada, on Jan. 4, 2023.
David Becker | Getty Images
AMD makes GPUs for gaming and, like Nvidia, is adapting them for AI inside of data centers. Its flagship chip is the Instinct MI300X. Microsoft has already bought AMD processors, offering access to them through its Azure cloud.
At launch, Su highlighted the chip’s excellence at inference, as opposed to competing with Nvidia for training. Last week, Microsoft said it was using AMD Instinct GPUs to serve its Copilot models. Morgan Stanley analysts took the news as a sign that AMD’s AI chip sales could surpass $4 billion this year, the company’s public target.
Intel, which was surpassed by Nvidia last year in terms of revenue, is also trying to establish a presence in AI. The company recently announced the third version of its AI accelerator, Gaudi 3. This time Intel compared it directly to the competition, describing it as a more cost-effective alternative and better than Nvidia’s H100 in terms of running inference, while faster at training models.
Bank of America analysts estimated recently that Intel will have less than 1% of the AI chip market this year. Intel says it has a $2 billion order of backlogs for the chip.
The main roadblock to broader adoption may be software. AMD and Intel are both participating in a big industry group called the UXL foundation, which includes Google, that’s working to create free alternatives to Nvidia’s CUDA for controlling hardware for AI applications.
Nvidia’s top customers
One potential challenge for Nvidia is that it’s competing against some of its biggest customers. Cloud providers including Google, Microsoft and Amazon are all building processors for internal use. The Big Tech three, plus Oracle, make up over 40% of Nvidia’s revenue.
Amazon introduced its own AI-oriented chips in 2018, under the Inferentia brand name. Inferentia is now on its second version. In 2021, Amazon Web Services debuted Tranium targeted to training. Customers can’t buy the chips but they can rent systems through AWS, which markets the chips as more cost efficient than Nvidia’s.
Google is perhaps the cloud provider most committed to its own silicon. The company has been using what it calls Tensor Processing Units (TPUs) since 2015 to train and deploy AI models. In May, Google announced the sixth version of its chip, Trillium, which the company said was used to develop its models, including Gemini and Imagen.
Google also uses Nvidia chips and offers them through its cloud.
Microsoft isn’t as far along. The company said last year that it was building its own AI accelerator and processor, called Maia and Cobalt.
Meta isn’t a cloud provider, but the company needs massive amounts of computing power to run its software and website and to serve ads. While the Facebook parent company is buying billions of dollars worth of Nvidia processors, it said in April that some of its homegrown chips were already in data centers and enabled “greater efficiency” compared to GPUs.
JPMorgan analysts estimated in May that the market for building custom chips for big cloud providers could be worth as much as $30 billion, with potential growth of 20% per year.
Startups
Cerebras’ WSE-3 chip is one example of new silicon from upstarts designed to run and train artificial intelligence.
Cerebras Systems
Venture capitalists see opportunities for emerging companies to jump into the game. They invested $6 billion in AI semiconductor companies in 2023, up slightly from $5.7 billion a year earlier, according to data from PitchBook.
It’s a tough area for startups as semiconductors are expensive to design, develop and manufacture. But there are opportunities for differentiation.
For Cerebras Systems, an AI chipmaker in Silicon Valley, the focus is on basic operations and bottlenecks for AI, versus the more general purpose nature of a GPU. The company was founded in 2015 and was valued at $4 billion during its most recent fundraising, according to Bloomberg.
The Cerebras chip, WSE-2, puts GPU capabilities as well as central processing and additional memory into a single device, which is better for training large models, said CEO Andrew Feldman.
“We use a giant chip, they use a lot of little chips,” Feldman said. “They’ve got challenges of moving data around, we don’t.”
Feldman said his company, which counts Mayo Clinic, GlaxoSmithKline, and the U.S. Military as clients, is winning business for its supercomputing systems even going up against Nvidia.
“There’s ample competition and I think that’s healthy for the ecosystem,” Feldman said.
Sheth from D-Matrix said his company plans to release a card with its chiplet later this year that will allow for more computation in memory, as opposed to on a chip like a GPU. D-Matrix’s product can be slotted into an AI server along existing GPUs, but it takes work off of Nvidia chips, and helps to lower the cost of generative AI.
Customers “are very receptive and very incentivized to enable a new solution to come to market,” Sheth said.
Apple and Qualcomm
Apple iPhone 15 series devices are displayed for sale at The Grove Apple retail store on release day in Los Angeles, California, on September 22, 2023.
Patrick T. Fallon | Afp | Getty Images
The biggest threat to Nvidia’s data center business may be a change in where processing happens.
Developers are increasingly betting that AI work will move from server farms to the laptops, PCs and phones we own.
Big models like the ones developed by OpenAI require massive clusters of powerful GPUs for inference, but companies like Apple and Microsoft are developing “small models” that require less power and data and can run on a battery-powered device. They may not be as skilled as the latest version of ChatGPT, but there are other applications they perform, such as summarizing text or visual search.
Apple and Qualcomm are updating their chips to run AI more efficiently, adding specialized sections for AI models called neural processors, which can have privacy and speed advantages.
Qualcomm recently announced a PC chip that will allow laptops to run Microsoft AI services on the device. The company has also invested in a number of chipmakers making lower-power processors to run AI algorithms outside of a smartphone or laptop.
Apple has been marketing its latest laptops and tablets as optimized for AI because of the neural engine on its chips. At its upcoming developer conference, Apple is planning to show off a slew of new AI features, likely running on the company’s iPhone-powering silicon.
Dado Ruvic | Reuters
If TikTok does indeed go dark on Sunday for Americans, there may be a tool for them to continue accessing the popular social app: VPNs.
The Chinese-owned app is set to be removed from mobile app stores and the web for U.S. users on Sunday as a result of a law signed by President Joe Biden in April 2024 requiring that the app be sold to a qualified buyer before the deadline.
Barring a last-minute sale or reprieve from the Supreme Court, the app will almost certainly vanish from the app stores for iPhones and Android phones. It won’t be removed from people’s phones, but the app could stop working.
TikTok plans to shut its service for Americans on Sunday, meaning that even those who already have the app downloaded won’t be able to continue using it, according to reports this week from Reuters and The Information. Apple and Google didn’t comment on their plans for taking down the apps from their app stores on Sunday.
“Basically, an app or a website can check where users came from,” said Justas Palekas, a head of product at IProyal.com, a proxy service. “Based on that, then they can impose restrictions based on their location.”
Masking your physical internet access point
That may stop most users, but for the particularly driven Americans, using VPNs might allow them to continue using the app.
VPNs and a related business-to-business technology called proxies work by tunneling a user’s internet traffic through a server in another country, making it look like they are accessing the internet from a location different than the one they are physically in.
This works because every time a computer connects to the internet, it is identified through an IP number, which is a 12-digit number that is different for every single computer. The first six digits of the number identifies the network, which also includes information about the physical region the request came from.
In China, people have used VPNs for years to get around the country’s firewall, which blocks U.S. websites such as Google and Facebook. VPNs saw big spikes in traffic when India banned TikTok in 2020, and people often use VPNs to watch sporting events from countries where official broadcasts aren’t available.
As of 2022, the VPN market was worth nearly $38 billion, according to the VPN Trust Initiative, a lobbying group.
“We consistently see significant spikes in VPN demand when access to online platforms is restricted, and this situation is no different,” said Lauren Hendry Parsons, privacy advocate at ExpressVPN, a VPN provider that costs $5 per month to use.
“We’re not here to endorse TikTok, but the looming U.S. ban highlights why VPNs matter— millions rely on them for secure, private, and unrestricted access to the internet,” ProtonVPN posted on social media earlier this week. ProtonVPN offers its service for $10 a month.
The price of VPNs
Both ExpressVPN and ProtonVPN allow users to set their internet-access location.
Most VPN services charge a monthly fee to pay for their servers and traffic, but some use a business model where they collect user data or traffic trends, such as when Meta offered a free VPN so it could keep an eye on which competitors’ apps were growing quickly.
A key tradeoff for those who use VPN is speed due to requests having to flow through a middleman computer to mask a users’ physical location.
And although VPNs have worked in the past when governments have banned apps, that doesn’t ensure that VPNs will work if TikTok goes dark. It won’t be clear if ExpressVPN would be able to access TikTok until after the ban takes place, Parsons told CNBC in an email. It’s also possible that TikTok may be able to determine Americans who try to use VPNs to access the app.
(L-R) Sarah Baus of Charleston, S.C., holds a sign that reads “Keep TikTok” as she and other content creators Sallye Miley of Jackson, Mississippi, and Callie Goodwin of Columbia, S.C., stand outside the U.S. Supreme Court Building as the court hears oral arguments on whether to overturn or delay a law that could lead to a ban of TikTok in the U.S., on January 10, 2025 in Washington, DC.
Andrew Harnik | Getty Images
VPNs and proxies to evade regional restrictions have been part of the internet’s landscape for decades, but their use is increasing as governments seek to ban certain services or apps.
Apps are removed by government request all the time. Nearly 1500 apps were removed in regions due to government takedown demands in 2023, according to Apple, with over 1,000 of them in China. Most of them are fringe apps that break laws such as those against gambling, or Chinese video game rules, but increasingly, countries are banning apps for national security or economic development reasons.
Now, the U.S. is poised to ban one of the most popular apps in the country — with 115 million users, it was the second most downloaded app of 2024 across both iOS and Android, according to an estimate provided to CNBC from Sensor Tower, a market intelligence firm.
“As we witness increasing attempts to fragment and censor the internet, the role of VPNs in upholding internet freedom is becoming increasingly critical,” Parsons said.
Technology
YouTube donating $15 million in LA wildfire relief, support for creators days before TikTok ban
Charred remains of buildings are pictured following the Palisades Fire in the Pacific Palisades neighborhood in Los Angeles, California, U.S. Jan. 15, 2025.
Mike Blake | Reuters
Google and YouTube will donate $15 million to support the Los Angeles community and content creators impacted by wildfires, YouTube CEO Neal Mohan announced in a blog post Wednesday.
The contributions will flow to local relief organizations including Emergency Network Los Angeles, the American Red Cross, the Center for Disaster Philanthropy and the Institute for Nonprofit News, the blog said. When the company’s LA offices can safely reopen, impacted creators will also be able to use YouTube’s production facilities “to recover and rebuild their businesses” as well as access community events.
“To all of our employees, the YouTube creator community, and everyone in LA, please stay safe and know we’re here to support,” Google CEO Sundar Pichai posted on X.
The move comes days before Sunday’s impending TikTok ban that has already seen content creators begin asking fans to follow them on other social platforms. YouTube Shorts, a short-form video platform within YouTube, is a competitor to TikTok, along with Meta’s Instagram Reels and the fast-growing Chinese app Rednote, otherwise known as Xiahongshu.
“In moments like these, we see the power of communities coming together to support each other — and the strength and resilience of the YouTube community is like no other,” Mohan wrote.
YouTube’s contributions are in line with a host of other LA companies pledging multi-million dollar donations aimed at assisting employees and residents impacted by the LA fires. Meta announced a $4 million donation split between CEO Mark Zuckerberg and the company while both Netflix and Comcast pledged $10 million donations to multiple aid groups.
Disclosure: Comcast owns NBCUniversal, the parent company of CNBC.
WATCH: TikTok: What creators would do if the short-form video app goes dark
Technology
TikTok’s U.S. operations could be worth as much as $50 billion if ByteDance decides to sell
Jakub Porzycki | Nurphoto | Getty Images
Business moguls such as Elon Musk should be prepared to spend tens of billions of dollars for TikTok’s U.S. operations should parent company ByteDance decide to sell.
TikTok is staring at a potential ban in the U.S. if the Supreme Court decides to uphold a national security law in which service providers such as Apple and Google would be penalized for hosting the app after the Sunday deadline. ByteDance has not indicated that it will sell the app’s U.S. unit, but the Chinese government has considered a plan in which X owner Musk would acquire the operations, as part of several scenarios in consideration, Bloomberg News reported Monday.
If ByteDance decides to sell, potential buyers may have to spend between $40 billion and $50 billion. That’s the valuation that CFRA Research Senior Vice President Angelo Zino has estimated for TikTok’s U.S. operations. Zino based his valuation on estimates of TikTok’s U.S. user base and revenue in comparison to rival apps.
TikTok has about 115 million monthly mobile users in the U.S., which is slightly behind Instagram’s 131 million, according to an estimate by market intelligence firm Sensor Tower. That puts TikTok ahead of Snapchat, Pinterest and Reddit, which have U.S. monthly mobile user bases of 96 million, 74 million and 32 million, according to Sensor Tower.
Zino’s estimate, however, is down from the more than $60 billion that he estimated for the unit in March 2024, when the House passed the initial national security bill that President Joe Biden signed into law the following month.
The lowered estimate is due to TikTok’s current geopolitical predicament and because “industry multiples have come in a bit” since March, Zino told CNBC in an email. Zino’s estimate doesn’t include TikTok’s valuable recommendation algorithms, which a U.S. acquirer would not obtain as part of a deal, with the algorithms and their alleged ties to China being central to the U.S. government’s case that TikTok poses a national security threat.
Analysts at Bloomberg Intelligence have their estimate for TikTok’s U.S. operations pegged in the range of $30 billion to $35 billion. That’s the estimate they published in July, saying at the time that the value of the unit would be “discounted due to it being a forced sale.”
Bloomberg Intelligence analysts noted that finding a buyer for TikTok’s U.S. operations that can both afford the transaction and deal with the accompanying regulatory scrutiny on data privacy makes a sale challenging. It could also make it difficult for a buyer to expand TikTok’s ads business, they wrote.
A consortium of businesspeople including billionaire Frank McCourt and O’Leary Ventures Chairman Kevin O’Leary put in a bid to buy TikTok from ByteDance. O’Leary has previously said the group would be willing to pay up to $20 billion to acquire the U.S. assets without the algorithm.
Unlike a Musk bid, O’Leary’s group’s bid would be free from regulatory scrutiny, O’Leary said in a Monday interview with Fox News.
O’Leary said that he’s “a huge Elon Musk fan,” but added “the idea that the regulator, even under Trump’s administration, would allow this is pretty slim.”
TikTok, X and O’Leary Ventures did not respond to requests for comment.
-
Sports2 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports9 months agoStory injured on diving stop, exits Red Sox game
-
Sports1 year agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports3 years ago
Team Europe easily wins 4th straight Laver Cup
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike
-
Business2 years agoBank of England’s extraordinary response to government policy is almost unthinkable | Ed Conway