Connect with us

Published

on

Nvidia CEO Jensen Huang speaks during a press conference at The MGM during CES 2018 in Las Vegas on January 7, 2018.

Mandel Ngan | AFP | Getty Images

Software that can write passages of text or draw pictures that look like a human created them has kicked off a gold rush in the technology industry.

Companies like Microsoft and Google are fighting to integrate cutting-edge AI into their search engines, as billion-dollar competitors such as OpenAI and Stable Diffusion race ahead and release their software to the public.

Powering many of these applications is a roughly $10,000 chip that’s become one of the most critical tools in the artificial intelligence industry: The Nvidia A100.

The A100 has become the “workhorse” for artificial intelligence professionals at the moment, said Nathan Benaich, an investor who publishes a newsletter and report covering the AI industry, including a partial list of supercomputers using A100s. Nvidia takes 95% of the market for graphics processors that can be used for machine learning, according to New Street Research.

A.I. is the catalyst behind Nividia's earnings beat, says Susquehanna's Christopher Rolland

The A100 is ideally suited for the kind of machine learning models that power tools like ChatGPT, Bing AI, or Stable Diffusion. It’s able to perform many simple calculations simultaneously, which is important for training and using neural network models.

The technology behind the A100 was initially used to render sophisticated 3D graphics in games. It’s often called a graphics processor, or GPU, but these days Nvidia’s A100 is configured and targeted at machine learning tasks and runs in data centers, not inside glowing gaming PCs.

Big companies or startups working on software like chatbots and image generators require hundreds or thousands of Nvidia’s chips, and either purchase them on their own or secure access to the computers from a cloud provider.

Hundreds of GPUs are required to train artificial intelligence models, like large language models. The chips need to be powerful enough to crunch terabytes of data quickly to recognize patterns. After that, GPUs like the A100 are also needed for “inference,” or using the model to generate text, make predictions, or identify objects inside photos.

This means that AI companies need access to a lot of A100s. Some entrepreneurs in the space even see the number of A100s they have access to as a sign of progress.

“A year ago we had 32 A100s,” Stability AI CEO Emad Mostaque wrote on Twitter in January. “Dream big and stack moar GPUs kids. Brrr.” Stability AI is the company that helped develop Stable Diffusion, an image generator that drew attention last fall, and reportedly has a valuation of over $1 billion.

Now, Stability AI has access to over 5,400 A100 GPUs, according to one estimate from the State of AI report, which charts and tracks which companies and universities have the largest collection of A100 GPUs — although it doesn’t include cloud providers, which don’t publish their numbers publicly.

Nvidia’s riding the A.I. train

Nvidia stands to benefit from the AI hype cycle. During Wednesday’s fiscal fourth-quarter earnings report, although overall sales declined 21%, investors pushed the stock up about 14% on Thursday, mainly because the company’s AI chip business — reported as data centers — rose by 11% to more than $3.6 billion in sales during the quarter, showing continued growth.

Nvidia shares are up 65% so far in 2023, outpacing the S&P 500 and other semiconductor stocks alike.

Nvidia CEO Jensen Huang couldn’t stop talking about AI on a call with analysts on Wednesday, suggesting that the recent boom in artificial intelligence is at the center of the company’s strategy.

“The activity around the AI infrastructure that we built, and the activity around inferencing using Hopper and Ampere to influence large language models has just gone through the roof in the last 60 days,” Huang said. “There’s no question that whatever our views are of this year as we enter the year has been fairly dramatically changed as a result of the last 60, 90 days.”

Ampere is Nvidia’s code name for the A100 generation of chips. Hopper is the code name for the new generation, including H100, which recently started shipping.

More computers needed

Nvidia A100 processor

Nvidia

Compared to other kinds of software, like serving a webpage, which uses processing power occasionally in bursts for microseconds, machine learning tasks can take up the whole computer’s processing power, sometimes for hours or days.

This means companies that find themselves with a hit AI product often need to acquire more GPUs to handle peak periods or improve their models.

These GPUs aren’t cheap. In addition to a single A100 on a card that can be slotted into an existing server, many data centers use a system that includes eight A100 GPUs working together.

This system, Nvidia’s DGX A100, has a suggested price of nearly $200,000, although it comes with the chips needed. On Wednesday, Nvidia said it would sell cloud access to DGX systems directly, which will likely reduce the entry cost for tinkerers and researchers.

It’s easy to see how the cost of A100s can add up.

For example, an estimate from New Street Research found that the OpenAI-based ChatGPT model inside Bing’s search could require 8 GPUs to deliver a response to a question in less than one second.

At that rate, Microsoft would need over 20,000 8-GPU servers just to deploy the model in Bing to everyone, suggesting Microsoft’s feature could cost $4 billion in infrastructure spending.

“If you’re from Microsoft, and you want to scale that, at the scale of Bing, that’s maybe $4 billion. If you want to scale at the scale of Google, which serves 8 or 9 billion queries every day, you actually need to spend $80 billion on DGXs.” said Antoine Chkaiban, a technology analyst at New Street Research. “The numbers we came up with are huge. But they’re simply the reflection of the fact that every single user taking to such a large language model requires a massive supercomputer while they’re using it.”

The latest version of Stable Diffusion, an image generator, was trained on 256 A100 GPUs, or 32 machines with 8 A100s each, according to information online posted by Stability AI, totaling 200,000 compute hours.

At the market price, training the model alone cost $600,000, Stability AI CEO Mostaque said on Twitter, suggesting in a tweet exchange the price was unusually inexpensive compared to rivals. That doesn’t count the cost of “inference,” or deploying the model.

Huang, Nvidia’s CEO, said in an interview with CNBC’s Katie Tarasov that the company’s products are actually inexpensive for the amount of computation that these kinds of models need.

“We took what otherwise would be a $1 billion data center running CPUs, and we shrunk it down into a data center of $100 million,” Huang said. “Now, $100 million, when you put that in the cloud and shared by 100 companies, is almost nothing.”

Huang said that Nvidia’s GPUs allow startups to train models for a much lower cost than if they used a traditional computer processor.

“Now you could build something like a large language model, like a GPT, for something like $10, $20 million,” Huang said. “That’s really, really affordable.”

New competition

Nvidia isn’t the only company making GPUs for artificial intelligence uses. AMD and Intel have competing graphics processors, and big cloud companies like Google and Amazon are developing and deploying their own chips specially designed for AI workloads.

Still, “AI hardware remains strongly consolidated to NVIDIA,” according to the State of AI compute report. As of December, more than 21,000 open-source AI papers said they used Nvidia chips.

Most researchers included in the State of AI Compute Index used the V100, Nvidia’s chip that came out in 2017, but A100 grew fast in 2022 to be the third-most used Nvidia chip, just behind a $1500-or-less consumer graphics chip originally intended for gaming.

The A100 also has the distinction of being one of only a few chips to have export controls placed on it because of national defense reasons. Last fall, Nvidia said in an SEC filing that the U.S. government imposed a license requirement barring the export of the A100 and the H100 to China, Hong Kong, and Russia.

“The USG indicated that the new license requirement will address the risk that the covered products may be used in, or diverted to, a ‘military end use’ or ‘military end user’ in China and Russia,” Nvidia said in its filing. Nvidia previously said it adapted some of its chips for the Chinese market to comply with U.S. export restrictions.

The fiercest competition for the A100 may be its successor. The A100 was first introduced in 2020, an eternity ago in chip cycles. The H100, introduced in 2022, is starting to be produced in volume — in fact, Nvidia recorded more revenue from H100 chips in the quarter ending in January than the A100, it said on Wednesday, although the H100 is more expensive per unit.

The H100, Nvidia says, is the first one of its data center GPUs to be optimized for transformers, an increasingly important technique that many of the latest and top AI applications use. Nvidia said on Wednesday that it wants to make AI training over 1 million percent faster. That could mean that, eventually, AI companies wouldn’t need so many Nvidia chips.

Continue Reading

Technology

How TikTok’s rise sparked a short-form video race

Published

on

By

How TikTok’s rise sparked a short-form video race

TikTok’s grip on the short-form video market is tightening, and the world’s biggest tech platforms are racing to catch up.

Since launching globally in 2016, ByteDance-owned TikTok has amassed over 1.12 billion monthly active users worldwide, according to Backlinko. American users spend an average of 108 minutes per day on the app, according to Apptoptia.

TikTok’s success has reshaped the social media landscape, forcing competitors like Meta and Google to pivot their strategies around short-form video. But so far, experts say that none have matched TikTok’s algorithmic precision.

“It is the center of the internet for young people,” said Jasmine Enberg, vice president and principal analyst at Emarketer. “It’s where they go for entertainment, news, trends, even shopping. TikTok sets the tone for everyone else.”

Platforms like Meta‘s Instagram Reels and Google’s YouTube Shorts have expanded aggressively, launching new features, creator tools and even considering separate apps just to compete. Microsoft-owned LinkedIn, traditionally a professional networking site, is the latest to experiment with TikTok-style feeds. But with TikTok continuing to evolve, adding features like e-commerce integrations and longer videos, the question remains whether rivals can keep up.

“I’m scrolling every single day. I doom scroll all the time,” said TikTok content creator Alyssa McKay.

But there may a dark side to this growth.

As short-form content consumption soars, experts warn about shrinking attention spans and rising mental-health concerns, particularly among younger users. Researchers like Dr. Yann Poncin, associate professor at the Child Study Center at Yale University, point to disrupted sleep patterns and increased anxiety levels tied to endless scrolling habits.

“Infinite scrolling and short-form video are designed to capture your attention in short bursts,” Dr. Poncin said. “In the past, entertainment was about taking you on a journey through a show or story. Now, it’s about locking you in for just a few seconds, just enough to feed you the next thing the algorithm knows you’ll like.”

Despite sky-high engagement, monetizing short videos remains an uphill battle. Unlike long-form YouTube content, where ads can be inserted throughout, short clips offer limited space for advertisers. Creators, too, are feeling the squeeze.

“It’s never been easier to go viral,” said Enberg. “But it’s never been harder to turn that virality into a sustainable business.”

Last year, TikTok generated an estimated $23.6 billion in ad revenues, according to Oberlo, but even with this growth, many creators still make just a few dollars per million views. YouTube Shorts pays roughly four cents per 1,000 views, which is less than its long-form counterpart. Meanwhile, Instagram has leaned into brand partnerships and emerging tools like “Trial Reels,” which allow creators to experiment with content by initially sharing videos only with non-followers, giving them a low-risk way to test new formats or ideas before deciding whether to share with their full audience. But Meta told CNBC that monetizing Reels remains a work in progress.

While lawmakers scrutinize TikTok’s Chinese ownership and explore potential bans, competitors see a window of opportunity. Meta and YouTube are poised to capture up to 50% of reallocated ad dollars if TikTok faces restrictions in the U.S., according to eMarketer.

Watch the video to understand how TikTok’s rise sparked a short form video race.

Continue Reading

Technology

Elon Musk’s xAI Holdings in talks to raise $20 billion, Bloomberg News reports

Published

on

By

Elon Musk's xAI Holdings in talks to raise  billion, Bloomberg News reports

The X logo appears on a phone, and the xAI logo is displayed on a laptop in Krakow, Poland, on April 1, 2025. (Photo by Klaudia Radecka/NurPhoto via Getty Images)

Nurphoto | Nurphoto | Getty Images

Elon Musk‘s xAI Holdings is in discussions with investors to raise about $20 billion, Bloomberg News reported Friday, citing people familiar with the matter.

The funding would value the company at over $120 billion, according to the report.

Musk was looking to assign “proper value” to xAI, sources told CNBC’s David Faber earlier this month. The remarks were made during a call with xAI investors, sources familiar with the matter told Faber. The Tesla CEO at that time didn’t explicitly mention any upcoming funding round, but the sources suggested xAI was preparing for a substantial capital raise in the near future.

The funding amount could be more than $20 billion as the exact figure had not been decided, the Bloomberg report added.

Artificial intelligence startup xAI didn’t immediately respond to a CNBC request for comment outside of U.S. business hours.

Faber Report: Elon Musk held call with current xAI investors, sources say

The AI firm last month acquired X in an all-stock deal that valued xAI at $80 billion and the social media platform at $33 billion.

“xAI and X’s futures are intertwined. Today, we officially take the step to combine the data, models, compute, distribution and talent,” Musk said on X, announcing the deal. “This combination will unlock immense potential by blending xAI’s advanced AI capability and expertise with X’s massive reach.”

Read the full Bloomberg story here.

— CNBC’s Samantha Subin contributed to this report.

Continue Reading

Technology

Alphabet jumps 3% as search, advertising units show resilient growth

Published

on

By

Alphabet jumps 3% as search, advertising units show resilient growth

Alphabet CEO Sundar Pichai during the Google I/O developers conference in Mountain View, California, on May 10, 2023.

David Paul Morris | Bloomberg | Getty Images

Alphabet‘s stock gained 3% Friday after signaling strong growth in its search and advertising businesses amid a competitive artificial intelligence environment and uncertain macro backdrop.

GOOGL‘s pace of GenAI product roll-out is accelerating with multiple encouraging signals,” wrote Morgan Stanley‘s Brian Nowak. “Macro uncertainty still exists but we remain [overweight] given GOOGL’s still strong relative position and improving pace of GenAI enabled product roll-out.”

The search giant posted earnings of $2.81 per share on $90.23 billion in revenues. That topped the $89.12 billion in sales and $2.01 in EPS expected by LSEG analysts. Revenues grew 12% year-over-year and ahead of the 10% anticipated by Wall Street.

Net income rose 46% to $34.54 billion, or $2.81 per share. That’s up from $23.66 billion, or $1.89 per share, in the year-ago period. Alphabet said the figure included $8 billion in unrealized gains on its nonmarketable equity securities connected to its investment in a private company.

Adjusted earnings, excluding that gain, were $2.27 per share, according to LSEG, and topped analyst expectations.

Read more CNBC tech news

Alphabet shares have pulled back about 16% this year as it battles volatility spurred by mounting trade war fears and worries that President Donald Trump‘s tariffs could crush the global economy. That would make it more difficult for Alphabet to potentially acquire infrastructure for data centers powering AI models as it faces off against competitors such as OpenAI and Anthropic to develop largely language models.

During Thursday’s call with investors, Alphabet suggested that it’s too soon to tally the total impact of tariffs. However, Google’s business chief Philipp Schindler said that ending the de minimis trade exemption in May, which created a loophole benefitting many Chinese e-commerce retailers, could create a “slight headwind” for the company’s ads business, specifically in the Asia-Pacific region. The loophole allows shipments under $800 to come into the U.S. duty-free.

Despite this backdrop, Alphabet showed steady growth in its advertising and search business, reporting $66.89 billion in revenues for its advertising unit. That reflected 8.5% growth from the year-ago period. The company reported $8.93 billion in advertising revenue for its YouTube business, shy of an $8.97 billion estimate from StreetAccount.

Alphabet’s “Search and other” unit rose 9.8% to $50.7 billion, up from $46.16 billion last year. The company said that its AI Overviews tool used in its Google search results page has accumulated 1.5 billion monthly users from a billion in October.

Bank of America analyst Justin Post said that Wall Street is underestimating the upside potential and “monetization ramp” from this tool and cloud demand fueled by AI.

“The strong 1Q search performance, along with constructive comments on Gemini [large language model] performance and [AI Overviews] adoption could help alleviate some investor concerns on AI competition,” Post wrote in a note.

WATCH: Gemini delivering well for Google, says Check Capital’s Chris Ballard

Gemini delivering well for Google, says Check Capital's Chris Ballard

CNBC’s Jennifer Elias contributed to this report.

Continue Reading

Trending