Nvidia unveils H200, its newest high-end chip for training AI models
Jensen Huang, president of Nvidia, holding the Grace hopper superchip CPU used for generative AI at the Supermicro keynote presentation during Computex 2023.
Walid Berrazeg | Lightrocket | Getty Images
Nvidia on Monday unveiled the H200, a graphics processing unit designed for training and deploying the kinds of artificial intelligence models that are powering the generative AI boom.
The new GPU is an upgrade from the H100, the chip OpenAI used to train its most advanced large language model, GPT-4. Big companies, startups and government agencies are all vying for a limited supply of the chips.
H100 chips cost between $25,000 and $40,000, according to an estimate from Raymond James, and thousands of them working together are needed to create the biggest models in a process called “training.”
Excitement over Nvidia’s AI GPUs has supercharged the company’s stock, which is up more than 230% so far in 2023. Nvidia expects around $16 billion of revenue for its fiscal third quarter, up 170% from a year ago.
The key improvement with the H200 is that it includes 141GB of next-generation “HBM3” memory that will help the chip perform “inference,” or using a large model after it’s trained to generate text, images or predictions.
Nvidia said the H200 will generate output nearly twice as fast as the H100. That’s based on a test using Meta’s Llama 2 LLM.
The H200, which is expected to ship in the second quarter of 2024, will compete with AMD’s MI300X GPU. AMD’s chip, similar to the H200, has additional memory over its predecessors, which helps fit big models on the hardware to run inference.
Nvidia H200 chips in an eight-GPU Nvidia HGX system.
Nvidia
Nvidia said the H200 will be compatible with the H100, meaning that AI companies who are already training with the prior model won’t need to change their server systems or software to use the new version.
Nvidia says it will be available in four-GPU or eight-GPU server configurations on the company’s HGX complete systems, as well as in a chip called GH200, which pairs the H200 GPU with an Arm-based processor.
However, the H200 may not hold the crown of the fastest Nvidia AI chip for long.
While companies like Nvidia offer many different configurations of their chips, new semiconductors often take a big step forward about every two years, when manufacturers move to a different architecture that unlocks more significant performance gains than adding memory or other smaller optimizations. Both the H100 and H200 are based on Nvidia’s Hopper architecture.
In October, Nvidia told investors that it would move from a two-year architecture cadence to a one-year release pattern due to high demand for its GPUs. The company displayed a slide suggesting it will announce and release its B100 chip, based on the forthcoming Blackwell architecture, in 2024.
WATCH: We’re a big believer in the AI trend going into next year
Don’t miss these stories from CNBC PRO:
Technology
South Korea’s LG Energy Solution signs $4.3 billion battery supply deal with undisclosed party
Technology
CyberArk’s stock jumps on report Palo Alto Networks in talks to buy company for over $20 billion
Nikesh Arora, CEO of Palo Alto Networks, looks on during the closing bell at the Nasdaq Market in New York City, U.S., March 25, 2025.
Jeenah Moon | Reuters
CyberArk shares soared as much as 18% on Tuesday after The Wall Street Journal reported that cybersecurity provider Palo Alto Networks has held discussions to buy the identity management software maker for over $20 billion.
Cloud security is becoming an increasingly critical piece of the enterprise tech stack, especially as rapid advancements in artificial intelligence bring with them a whole new set of threats, and as ransomware attacks become more commonplace.
Founded in 2005, Palo Alto Networks has emerged in recent years as a consolidator in the cybersecurity industry and has grown into the biggest player in the space by market cap, with a valuation of over $130 billion. CEO Nikesh Arora, who was appointed to the job in 2018, has been on a spending spree, snapping up Protect AI in a deal that closed in July, and in 2023 buying Talon Cyber Security, Dig Security and Zycada Networks.
But CyberArk would represent by far Arora’s biggest bet yet. The Israeli company, which went public in 2014, provides technology that helps companies streamline the process of logging on to applications for employees.
CyberArk faces competition from Microsoft, Okta and IBM‘s HashiCorp. Another rival, SailPoint, returned to the public markets in February.
With Tuesday’s rally, CyberArk shares climbed to a record, surpassing their prior all-time high reached in February. The stock is up 29% this year, pushing the company’s market cap to almost $21 billion, after jumping 52% in 2024. Palo Alto shares, meanwhile, slid 3.5% on the report and are now up about 9% for the year.
Representatives from Palo Alto Networks and CyberArk declined to comment.
During the first quarter, CyberArk generated around $11.5 million in net income on around $318 million in revenue, which was up 43% from a year earlier.
It’s been an active stretch for big deals in the cyber market. Google said in March that it was spending $32 billion on Wiz, its largest acquisition on record by far, and a purchase intended to bolster its cloud business with greater AI security technology.
Networking giant Cisco also made its biggest deal ever in the security space, buying Splunk in 2023 for $28 billion. Splunk’s technology helps businesses monitor and analyze their data to minimize the risk of hacks and resolve technical issues faster.
— CNBC’s Ari Levy contributed to this report
-
Sports3 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoStory injured on diving stop, exits Red Sox game
-
Sports2 years agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Sports2 years agoButton battles heat exhaustion in NASCAR debut
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike