Connect with us

Published

on

Nvidia CEO Jensen Huang speaks during a press conference at The MGM during CES 2018 in Las Vegas on January 7, 2018.

Mandel Ngan | AFP | Getty Images

Software that can write passages of text or draw pictures that look like a human created them has kicked off a gold rush in the technology industry.

Companies like Microsoft and Google are fighting to integrate cutting-edge AI into their search engines, as billion-dollar competitors such as OpenAI and Stable Diffusion race ahead and release their software to the public.

Powering many of these applications is a roughly $10,000 chip that’s become one of the most critical tools in the artificial intelligence industry: The Nvidia A100.

The A100 has become the “workhorse” for artificial intelligence professionals at the moment, said Nathan Benaich, an investor who publishes a newsletter and report covering the AI industry, including a partial list of supercomputers using A100s. Nvidia takes 95% of the market for graphics processors that can be used for machine learning, according to New Street Research.

A.I. is the catalyst behind Nividia's earnings beat, says Susquehanna's Christopher Rolland

The A100 is ideally suited for the kind of machine learning models that power tools like ChatGPT, Bing AI, or Stable Diffusion. It’s able to perform many simple calculations simultaneously, which is important for training and using neural network models.

The technology behind the A100 was initially used to render sophisticated 3D graphics in games. It’s often called a graphics processor, or GPU, but these days Nvidia’s A100 is configured and targeted at machine learning tasks and runs in data centers, not inside glowing gaming PCs.

Big companies or startups working on software like chatbots and image generators require hundreds or thousands of Nvidia’s chips, and either purchase them on their own or secure access to the computers from a cloud provider.

Hundreds of GPUs are required to train artificial intelligence models, like large language models. The chips need to be powerful enough to crunch terabytes of data quickly to recognize patterns. After that, GPUs like the A100 are also needed for “inference,” or using the model to generate text, make predictions, or identify objects inside photos.

This means that AI companies need access to a lot of A100s. Some entrepreneurs in the space even see the number of A100s they have access to as a sign of progress.

“A year ago we had 32 A100s,” Stability AI CEO Emad Mostaque wrote on Twitter in January. “Dream big and stack moar GPUs kids. Brrr.” Stability AI is the company that helped develop Stable Diffusion, an image generator that drew attention last fall, and reportedly has a valuation of over $1 billion.

Now, Stability AI has access to over 5,400 A100 GPUs, according to one estimate from the State of AI report, which charts and tracks which companies and universities have the largest collection of A100 GPUs — although it doesn’t include cloud providers, which don’t publish their numbers publicly.

Nvidia’s riding the A.I. train

Nvidia stands to benefit from the AI hype cycle. During Wednesday’s fiscal fourth-quarter earnings report, although overall sales declined 21%, investors pushed the stock up about 14% on Thursday, mainly because the company’s AI chip business — reported as data centers — rose by 11% to more than $3.6 billion in sales during the quarter, showing continued growth.

Nvidia shares are up 65% so far in 2023, outpacing the S&P 500 and other semiconductor stocks alike.

Nvidia CEO Jensen Huang couldn’t stop talking about AI on a call with analysts on Wednesday, suggesting that the recent boom in artificial intelligence is at the center of the company’s strategy.

“The activity around the AI infrastructure that we built, and the activity around inferencing using Hopper and Ampere to influence large language models has just gone through the roof in the last 60 days,” Huang said. “There’s no question that whatever our views are of this year as we enter the year has been fairly dramatically changed as a result of the last 60, 90 days.”

Ampere is Nvidia’s code name for the A100 generation of chips. Hopper is the code name for the new generation, including H100, which recently started shipping.

More computers needed

Nvidia A100 processor

Nvidia

Compared to other kinds of software, like serving a webpage, which uses processing power occasionally in bursts for microseconds, machine learning tasks can take up the whole computer’s processing power, sometimes for hours or days.

This means companies that find themselves with a hit AI product often need to acquire more GPUs to handle peak periods or improve their models.

These GPUs aren’t cheap. In addition to a single A100 on a card that can be slotted into an existing server, many data centers use a system that includes eight A100 GPUs working together.

This system, Nvidia’s DGX A100, has a suggested price of nearly $200,000, although it comes with the chips needed. On Wednesday, Nvidia said it would sell cloud access to DGX systems directly, which will likely reduce the entry cost for tinkerers and researchers.

It’s easy to see how the cost of A100s can add up.

For example, an estimate from New Street Research found that the OpenAI-based ChatGPT model inside Bing’s search could require 8 GPUs to deliver a response to a question in less than one second.

At that rate, Microsoft would need over 20,000 8-GPU servers just to deploy the model in Bing to everyone, suggesting Microsoft’s feature could cost $4 billion in infrastructure spending.

“If you’re from Microsoft, and you want to scale that, at the scale of Bing, that’s maybe $4 billion. If you want to scale at the scale of Google, which serves 8 or 9 billion queries every day, you actually need to spend $80 billion on DGXs.” said Antoine Chkaiban, a technology analyst at New Street Research. “The numbers we came up with are huge. But they’re simply the reflection of the fact that every single user taking to such a large language model requires a massive supercomputer while they’re using it.”

The latest version of Stable Diffusion, an image generator, was trained on 256 A100 GPUs, or 32 machines with 8 A100s each, according to information online posted by Stability AI, totaling 200,000 compute hours.

At the market price, training the model alone cost $600,000, Stability AI CEO Mostaque said on Twitter, suggesting in a tweet exchange the price was unusually inexpensive compared to rivals. That doesn’t count the cost of “inference,” or deploying the model.

Huang, Nvidia’s CEO, said in an interview with CNBC’s Katie Tarasov that the company’s products are actually inexpensive for the amount of computation that these kinds of models need.

“We took what otherwise would be a $1 billion data center running CPUs, and we shrunk it down into a data center of $100 million,” Huang said. “Now, $100 million, when you put that in the cloud and shared by 100 companies, is almost nothing.”

Huang said that Nvidia’s GPUs allow startups to train models for a much lower cost than if they used a traditional computer processor.

“Now you could build something like a large language model, like a GPT, for something like $10, $20 million,” Huang said. “That’s really, really affordable.”

New competition

Nvidia isn’t the only company making GPUs for artificial intelligence uses. AMD and Intel have competing graphics processors, and big cloud companies like Google and Amazon are developing and deploying their own chips specially designed for AI workloads.

Still, “AI hardware remains strongly consolidated to NVIDIA,” according to the State of AI compute report. As of December, more than 21,000 open-source AI papers said they used Nvidia chips.

Most researchers included in the State of AI Compute Index used the V100, Nvidia’s chip that came out in 2017, but A100 grew fast in 2022 to be the third-most used Nvidia chip, just behind a $1500-or-less consumer graphics chip originally intended for gaming.

The A100 also has the distinction of being one of only a few chips to have export controls placed on it because of national defense reasons. Last fall, Nvidia said in an SEC filing that the U.S. government imposed a license requirement barring the export of the A100 and the H100 to China, Hong Kong, and Russia.

“The USG indicated that the new license requirement will address the risk that the covered products may be used in, or diverted to, a ‘military end use’ or ‘military end user’ in China and Russia,” Nvidia said in its filing. Nvidia previously said it adapted some of its chips for the Chinese market to comply with U.S. export restrictions.

The fiercest competition for the A100 may be its successor. The A100 was first introduced in 2020, an eternity ago in chip cycles. The H100, introduced in 2022, is starting to be produced in volume — in fact, Nvidia recorded more revenue from H100 chips in the quarter ending in January than the A100, it said on Wednesday, although the H100 is more expensive per unit.

The H100, Nvidia says, is the first one of its data center GPUs to be optimized for transformers, an increasingly important technique that many of the latest and top AI applications use. Nvidia said on Wednesday that it wants to make AI training over 1 million percent faster. That could mean that, eventually, AI companies wouldn’t need so many Nvidia chips.

Continue Reading

Technology

Super Micro plans to ramp up manufacturing in Europe to capitalize on AI demand

Published

on

By

Super Micro plans to ramp up manufacturing in Europe to capitalize on AI demand

CEO of Supermicro Charles Liang speaks during the Reuters NEXT conference in New York City, U.S., December 10, 2024. 

Mike Segar | Reuters

PARIS — Super Micro plans to increase its investment in Europe, including ramping up manufacturing of its AI servers in the region, CEO Charles Liang told CNBC in an interview that aired on Wednesday.

The company sells servers which are packed with Nvidia chips and are key for training and implementing huge AI models. It has manufacturing facilities in the Netherlands, but could expand to other places.

“But because the demand in Europe is growing very fast, so I already decided, indeed, [there’s] already a plan to invest more in Europe, including manufacturing,” Liang told CNBC at the Raise Summit in Paris, France.

“The demand is global, and the demand will continue to improve in [the] next many years,” Liang added.

Liang’s comments come less than a month after Nvidia CEO Jensen Huang visited various parts of Europe, signing infrastructure deals and urging the region to ramp up its computing capacity.

Growth to be ‘strong’

Super Micro rode the growth wave after OpenAI’s ChatGPT boom boosted demand for Nvidia’s chips, which underpin big AI models. The server maker’s stock hit a record high in March 2024. However, the stock is around 60% off that all-time high over concerns about its accounting and financial reporting. But the company in February filed its delayed financial report for its 2024 fiscal year, assuaging those fears.

In May, the company reported weaker-than-expected guidance for the current quarter, raising concerns about demand for its product.

However, Liang dismissed those fears. “Our growth rate continues to be strong, because we continue to grow our fundamental technology, and we [are] also expanding our business scope,” Liang said.

“So the room … to grow will be still very tremendous, very big.”

Continue Reading

Technology

Apple says COO Jeff Williams will retire from company later this year

Published

on

By

Apple says COO Jeff Williams will retire from company later this year

Jeff Williams, chief operating officer of Apple Inc., during the Apple Worldwide Developers Conference (WWDC) at Apple Park campus in Cupertino, California, US, on Monday, June 9, 2025.

David Paul Morris | Bloomberg | Getty Images

Apple said on Tuesday that Chief Operating Officer Jeff Williams, a 27-year company veteran, will be retiring later this year.

Current operations leader Sabih Khan will take over much of the COO role later this month, Apple said in a press release. For his remaining time with the comapny, Williams will continue to head up Apple’s design team, Apple Watch, and health initiatives, reporting to CEO Tim Cook.

Williams becomes the latest longtime Apple executive to step down as key employees, who were active in the company’s hyper-growth years, reach retirement age. Williams, 62, previously headed Apple’s formidable operations division, which is in charge of manufacturing millions of complicated devices like iPhones, while keeping costs down.

He also led important teams inside Apple, including the company’s fabled industrial design team, after longtime leader Jony Ive retired in 2019. When Williams retires, Apple’s design team will report to CEO Tim Cook, Apple said.

“He’s helped to create one of the most respected global supply chains in the world; launched Apple Watch and overseen its development; architected Apple’s health strategy; and led our world class team of designers with great wisdom, heart, and dedication,” Cook said in the statement.

Williams said he plans to spend more time with friends and family.

“June marked my 27th anniversary with Apple, and my 40th in the industry,” Williams said in the release.

Williams is leaving Apple at a time when its famous supply chain is under significant pressure, as the U.S. imposes tariffs on many of the countries where Apple sources its devices, and White House officials publicly pressure Apple to move more production to the U.S.

Khan was added to Apple’s executive team in 2019, taking an executive vice president title. Apple said on Tuesday that he will lead supply chain, product quality, planning, procurement, and fulfillment at Apple.

The operations leader joined Apple’s procurement group in 1995, and before that worked as an engineer and technical leader at GE Plastics. He has a bachelor’s degree from Tufts University and a master’s degree in mechanical engineering from Rensselaer Polytechnic Institute in upstate New York.

Khan has worked closely with Cook. Once, during a meeting when Cook said that a manufacturing problem was “really bad,” Khan stood up and drove to the airport, and immediately booked a flight to China to fix it, according to an anecdote published in Fortune.

WATCH: Jefferies upgrades Apple

Jefferies upgrades Apple to 'Hold'

Continue Reading

Technology

Elon Musk lashes out at Tesla bull Dan Ives over board proposals: ‘Shut up’

Published

on

By

Elon Musk lashes out at Tesla bull Dan Ives over board proposals: 'Shut up'

Elon Musk, chief executive officer of SpaceX and Tesla, attends the Viva Technology conference at the Porte de Versailles exhibition center in Paris, June 16, 2023.

Gonzalo Fuentes | Reuters

Tesla CEO Elon Musk told Wedbush Securities’ Dan Ives to “Shut up” on Tuesday after the analyst offered three recommendations to the electric vehicle company’s board in a post on X.

Ives has been one of the most bullish Tesla observers on Wall Street. With a $500 price target on the stock, he has the highest projection of any analyst tracked by FactSet.

But on Tuesday, Ives took to X with critical remarks about Musk’s political activity after the world’s richest person said over the weekend that he was creating a new political party called the America Party to challenge Republican candidates who voted for the spending bill that was backed by President Donald Trump.

Ives’ post followed a nearly 7% slide in Tesla’s stock Monday, which wiped out $68 billion in market cap. Ives called for Tesla’s board to create a new pay package for Musk that would get him 25% voting control and clear a path to merge with xAI, establish “guardrails” for how much time Musk has to spend at Tesla, and provide “oversight on political endeavors.”

Ives published a lengthier note with other analysts from his firm headlined, “The Tesla board MUST Act and Create Ground Rules For Musk; Soap Opera Must End.” The analysts said that Musk’s launching of a new political party created a “tipping point in the Tesla story,” necessitating action by the company’s board to rein in the CEO.

Still, Wedbush maintained its price target and its buy recommendation on the stock.

“Shut up, Dan,” Musk wrote in response on X, even though the first suggestion would hand the CEO the voting control he has long sought at Tesla.

In an email to CNBC, Ives wrote, “Elon has his opinion and I get it, but we stand by what the right course of action is for the Board.”

Musk’s historic 2018 CEO pay package, which had been worth around $56 billion and has since gone up in value, was voided last year by the Delaware Court of Chancery. Judge Kathaleen McCormick ruled that Tesla’s board members had lacked independence from Musk and failed to properly negotiate at arm’s length with the CEO.

Elon Musk can't continue to go down this political path, says Wedbush's Dan Ives

Tesla has appealed that case to the Delaware state Supreme Court and is trying to determine what Musk’s next pay package should entail.

Ives isn’t the only Tesla bull to criticize Musk’s continued political activism.

Analysts at William Blair downgraded the stock to the equivalent of a hold from a buy on Monday, because of Musk’s political plans and rhetoric as well as the negative impacts that the spending bill passed by Congress could have on Tesla’s margins and EV sales.

“We expect that investors are growing tired of the distraction at a point when the business needs Musk’s attention the most and only see downside from his dip back into politics,” the analysts wrote. “We would prefer this effort to be channeled towards the robotaxi rollout at this critical juncture.”

Trump supporter James Fishback, CEO of hedge fund Azoria Partners, said Saturday that his firm postponed the listing of an exchange-traded fund, the Azoria Tesla Convexity ETF, that would invest in the EV company’s shares and options. He began his post on X saying, “Elon has gone too far.”

“I encourage the Board to meet immediately and ask Elon to clarify his political ambitions and evaluate whether they are compatible with his full-time obligations to Tesla as CEO,” Fishback wrote.

Musk said Saturday that he has formed the America Party, which he claimed will give Americans “back your freedom.” He hasn’t shared formal details, including where the party may be registered, how much funding he will provide for it and which candidates he will back.

Tesla’s stock is now down about 25% this year, badly underperforming U.S. indexes and by far the worst performance among tech’s megacaps.

Musk spent much of the first half of the year working with the Trump administration and leading an effort to massively downsize the federal government. His official work with the administration wrapped up at the end of May, and his exit preceded a public spat between Musk and Trump over the spending bill and other matters.

Musk, Tesla’s board chair Robyn Denholm and investor relations representative Travis Axelrod didn’t immediately respond to requests for comment.

WATCH: Musk-backed party would be doomed by his unfavorability

Musk-backed party would be doomed by his unfavorability, says Big Technology's Alex Kantrowitz

Continue Reading

Trending