Connect with us

Published

on

Jensen Huang, president of Nvidia, holding the Grace hopper superchip CPU used for generative AI at the Supermicro keynote presentation during Computex 2023.

Walid Berrazeg | Lightrocket | Getty Images

Nvidia on Monday unveiled the H200, a graphics processing unit designed for training and deploying the kinds of artificial intelligence models that are powering the generative AI boom.

The new GPU is an upgrade from the H100, the chip OpenAI used to train its most advanced large language model, GPT-4. Big companies, startups and government agencies are all vying for a limited supply of the chips.

H100 chips cost between $25,000 and $40,000, according to an estimate from Raymond James, and thousands of them working together are needed to create the biggest models in a process called “training.”

Excitement over Nvidia’s AI GPUs has supercharged the company’s stock, which is up more than 230% so far in 2023. Nvidia expects around $16 billion of revenue for its fiscal third quarter, up 170% from a year ago.

The key improvement with the H200 is that it includes 141GB of next-generation “HBM3” memory that will help the chip perform “inference,” or using a large model after it’s trained to generate text, images or predictions.

Nvidia said the H200 will generate output nearly twice as fast as the H100. That’s based on a test using Meta’s Llama 2 LLM.

The H200, which is expected to ship in the second quarter of 2024, will compete with AMD’s MI300X GPU. AMD’s chip, similar to the H200, has additional memory over its predecessors, which helps fit big models on the hardware to run inference.

Nvidia H200 chips in an eight-GPU Nvidia HGX system.

Nvidia

Nvidia said the H200 will be compatible with the H100, meaning that AI companies who are already training with the prior model won’t need to change their server systems or software to use the new version.

Nvidia says it will be available in four-GPU or eight-GPU server configurations on the company’s HGX complete systems, as well as in a chip called GH200, which pairs the H200 GPU with an Arm-based processor.

However, the H200 may not hold the crown of the fastest Nvidia AI chip for long.

While companies like Nvidia offer many different configurations of their chips, new semiconductors often take a big step forward about every two years, when manufacturers move to a different architecture that unlocks more significant performance gains than adding memory or other smaller optimizations. Both the H100 and H200 are based on Nvidia’s Hopper architecture.

In October, Nvidia told investors that it would move from a two-year architecture cadence to a one-year release pattern due to high demand for its GPUs. The company displayed a slide suggesting it will announce and release its B100 chip, based on the forthcoming Blackwell architecture, in 2024.

WATCH: We’re a big believer in the AI trend going into next year

We're a big believer in the A.I. trend going into next year, says Wolfe Research's Chris Caso

Don’t miss these stories from CNBC PRO:

Continue Reading

Technology

AI infrastructure startup CoreWeave raises $7.5 billion in debt deal led by Blackstone

Published

on

By

AI infrastructure startup CoreWeave raises .5 billion in debt deal led by Blackstone

Michael Intrator, CEO of CoreWeave, participates in a CNBC interview on May 9, 2024.

CNBC

Fresh off a $1.1 billion equity funding round, artificial intelligence infrastructure startup CoreWeave has raised $7.5 billion in debt so that it can more heavily invest in its cloud data centers.

Blackstone’s funds led the lending round, with participation from Coatue, Carlyle, BlackRock and others. In its equity financing two weeks ago, CoreWeave was valued at $19 billion.

Investors are flocking to CoreWeave, because the 550-person company is one of the main providers of Nvidia’s chips for running AI models. Demand for the technology is soaring as businesses across virtually all sectors are racing to integrate AI chatbots into their products following the launch of OpenAI’s ChatGPT in late 2022.

With Nvidia’s AI-focused graphics processing units (GPUs) in limited supply, CoreWeave’s access to the processors has made it a hot commodity. That means the company, which is backed by Nvidia, is going up against the world’s top cloud infrastructure operators, including Amazon and Google.

On its website, CoreWeave claims to have lower on-demand prices than any major cloud company. Even Microsoft, the world’s second-largest provider of cloud infrastructure, has started relying on CoreWeave to help supply OpenAI with the computing power it needs.

Collette Kress, Nvidia’s finance chief, said at a Citigroup event in September that CoreWeave has “quite some skills in terms of just their speed of adoption, their speed in terms of setting things up.”

A CoreWeave spokesperson declined to comment on whether the company is using Nvidia GPUs as collateral for the fresh debt financing. Such GPUs were used as collateral in a $2.3 billion debt round last year, Reuters reported.

The new debt will help CoreWeave pay for servers loaded with GPUs, as well as networking equipment and cabinets, the spokesperson said.

WATCH: CoreWeave CEO Michael Intrator discusses the competitive landscape

CoreWeave CEO Michael Intrator discusses the competitive AI landscape

Continue Reading

Technology

Microsoft’s Mistral partnership avoids merger probe by British regulators

Published

on

By

Microsoft's Mistral partnership avoids merger probe by British regulators

The Microsoft logo is displayed on a smartphone.

Mateusz Slodkowski | Sopa Images | Lightrocket | Getty Images

The U.K.’s Competition and Markets Authority cleared Microsoft’s AI partnership with Mistral of regulatory concerns after previously inviting views on whether the arrangement qualified as a merger.

The CMA said in a brief statement Friday that the deal “does not qualify for investigation under the merger provisions of the Enterprise Act 2002.”

CNBC has reached out to Microsoft and Mistral.

Mistral, a French AI firm founded in 2023, won a 15 million euro ($16 million) investment from Microsoft earlier this year.

Under the terms of the deal, the U.S. tech giant receives a minority stake in Mistral, while the French company adds its large language models to the U.S. tech giant’s Azure cloud computing platform.

In April, the CMA began seeking views from interested parties on partnerships agreed by U.S. tech giants with smaller AI firms to determine whether arrangements between the companies qualify as mergers.

As part of that effort, the CMA looked into the minority investment deals agreed by Microsoft and Mistral, as well as into whether Microsoft’s hiring of certain former employees from AI startup Inflection constitutes a merger. The watchdog separately invited comment on the arrangements between Amazon and Anthropic.

Now, the regulator says it’s no longer looking into Microsoft’s investment in Mistral. It has given no update on its inquiries into the Amazon-Inflection deal and into Microsoft’s hiring of employees from Inflection.

Microsoft previously denied its deals with OpenAI and Mistral and hiring of employees from Inflection constituted mergers. Amazon has also said that its partnership with Anthropic represents a limited corporate investment, not a merger.

Continue Reading

Technology

Chinese EV maker Xpeng aims to deliver its first flying car in 2026

Published

on

By

Chinese EV maker Xpeng aims to deliver its first flying car in 2026

Xpeng’s “Land Aircraft Carrier” vehicle has a flying passenger drone hidden inside the truck. The drone can detach.

XPENG AEROHT

Xpeng AeroHT, an affilaite of Xpeng, aims to deliver its flying car to customers in 2026, the Chinese electric vehicle maker’s co-president told CNBC on Friday,

Last year, Xpeng AeroHT introduced the Land Aircraft Carrier — a large truck with a flying two-seater passenger electric drone inside. The flying car can detach from the truck, and people can then get into the drone and fly it.

Brian Gu, co-president of Xpeng, said the vehicle will be available for pre-order this year, adding that the company hopes to deliver the unit in 2026.

Chinese auto giant Xpeng wants to deliver flying cars by 2026

“The reason we are confident, because we are designing this for the use not in urban centers, but for outskirts in scenic areas where … we will work with municipalities to create flying parks and flying zones that allow people to enjoy flying without the hassle of getting all the complicated approvals,” Gu noted.

Xpeng said this year that the flying car is currently going through a certification process with the Chinese aviation regulator.

The 2026 timeline is slightly later than the fourth-quarter 2025 delivery target that Xpeng had previously touted.

Gu said passengers will not require a special license to fly the drone for initial use.

“Because we are using leisure and sports related use case for the initial use of that flying device. As you move more closer to urban … centers, you do need special licenses and that will be a lot more complicated to get approval for,” Gu said.

Xpeng has been looking to expand into other areas of electric mobility, with company CEO He Xiaopeng previously telling CNBC that robotics and flying cars were part of the company’s longer-term goals.

eVTOLS: Are flying cars finally becoming reality?

Continue Reading

Trending