Connect with us

Published

on

Nvidia CEO Jensen Huang speaks during a press conference at The MGM during CES 2018 in Las Vegas on January 7, 2018.

Mandel Ngan | AFP | Getty Images

Software that can write passages of text or draw pictures that look like a human created them has kicked off a gold rush in the technology industry.

Companies like Microsoft and Google are fighting to integrate cutting-edge AI into their search engines, as billion-dollar competitors such as OpenAI and Stable Diffusion race ahead and release their software to the public.

Powering many of these applications is a roughly $10,000 chip that’s become one of the most critical tools in the artificial intelligence industry: The Nvidia A100.

The A100 has become the “workhorse” for artificial intelligence professionals at the moment, said Nathan Benaich, an investor who publishes a newsletter and report covering the AI industry, including a partial list of supercomputers using A100s. Nvidia takes 95% of the market for graphics processors that can be used for machine learning, according to New Street Research.

A.I. is the catalyst behind Nividia's earnings beat, says Susquehanna's Christopher Rolland

The A100 is ideally suited for the kind of machine learning models that power tools like ChatGPT, Bing AI, or Stable Diffusion. It’s able to perform many simple calculations simultaneously, which is important for training and using neural network models.

The technology behind the A100 was initially used to render sophisticated 3D graphics in games. It’s often called a graphics processor, or GPU, but these days Nvidia’s A100 is configured and targeted at machine learning tasks and runs in data centers, not inside glowing gaming PCs.

Big companies or startups working on software like chatbots and image generators require hundreds or thousands of Nvidia’s chips, and either purchase them on their own or secure access to the computers from a cloud provider.

Hundreds of GPUs are required to train artificial intelligence models, like large language models. The chips need to be powerful enough to crunch terabytes of data quickly to recognize patterns. After that, GPUs like the A100 are also needed for “inference,” or using the model to generate text, make predictions, or identify objects inside photos.

This means that AI companies need access to a lot of A100s. Some entrepreneurs in the space even see the number of A100s they have access to as a sign of progress.

“A year ago we had 32 A100s,” Stability AI CEO Emad Mostaque wrote on Twitter in January. “Dream big and stack moar GPUs kids. Brrr.” Stability AI is the company that helped develop Stable Diffusion, an image generator that drew attention last fall, and reportedly has a valuation of over $1 billion.

Now, Stability AI has access to over 5,400 A100 GPUs, according to one estimate from the State of AI report, which charts and tracks which companies and universities have the largest collection of A100 GPUs — although it doesn’t include cloud providers, which don’t publish their numbers publicly.

Nvidia’s riding the A.I. train

Nvidia stands to benefit from the AI hype cycle. During Wednesday’s fiscal fourth-quarter earnings report, although overall sales declined 21%, investors pushed the stock up about 14% on Thursday, mainly because the company’s AI chip business — reported as data centers — rose by 11% to more than $3.6 billion in sales during the quarter, showing continued growth.

Nvidia shares are up 65% so far in 2023, outpacing the S&P 500 and other semiconductor stocks alike.

Nvidia CEO Jensen Huang couldn’t stop talking about AI on a call with analysts on Wednesday, suggesting that the recent boom in artificial intelligence is at the center of the company’s strategy.

“The activity around the AI infrastructure that we built, and the activity around inferencing using Hopper and Ampere to influence large language models has just gone through the roof in the last 60 days,” Huang said. “There’s no question that whatever our views are of this year as we enter the year has been fairly dramatically changed as a result of the last 60, 90 days.”

Ampere is Nvidia’s code name for the A100 generation of chips. Hopper is the code name for the new generation, including H100, which recently started shipping.

More computers needed

Nvidia A100 processor

Nvidia

Compared to other kinds of software, like serving a webpage, which uses processing power occasionally in bursts for microseconds, machine learning tasks can take up the whole computer’s processing power, sometimes for hours or days.

This means companies that find themselves with a hit AI product often need to acquire more GPUs to handle peak periods or improve their models.

These GPUs aren’t cheap. In addition to a single A100 on a card that can be slotted into an existing server, many data centers use a system that includes eight A100 GPUs working together.

This system, Nvidia’s DGX A100, has a suggested price of nearly $200,000, although it comes with the chips needed. On Wednesday, Nvidia said it would sell cloud access to DGX systems directly, which will likely reduce the entry cost for tinkerers and researchers.

It’s easy to see how the cost of A100s can add up.

For example, an estimate from New Street Research found that the OpenAI-based ChatGPT model inside Bing’s search could require 8 GPUs to deliver a response to a question in less than one second.

At that rate, Microsoft would need over 20,000 8-GPU servers just to deploy the model in Bing to everyone, suggesting Microsoft’s feature could cost $4 billion in infrastructure spending.

“If you’re from Microsoft, and you want to scale that, at the scale of Bing, that’s maybe $4 billion. If you want to scale at the scale of Google, which serves 8 or 9 billion queries every day, you actually need to spend $80 billion on DGXs.” said Antoine Chkaiban, a technology analyst at New Street Research. “The numbers we came up with are huge. But they’re simply the reflection of the fact that every single user taking to such a large language model requires a massive supercomputer while they’re using it.”

The latest version of Stable Diffusion, an image generator, was trained on 256 A100 GPUs, or 32 machines with 8 A100s each, according to information online posted by Stability AI, totaling 200,000 compute hours.

At the market price, training the model alone cost $600,000, Stability AI CEO Mostaque said on Twitter, suggesting in a tweet exchange the price was unusually inexpensive compared to rivals. That doesn’t count the cost of “inference,” or deploying the model.

Huang, Nvidia’s CEO, said in an interview with CNBC’s Katie Tarasov that the company’s products are actually inexpensive for the amount of computation that these kinds of models need.

“We took what otherwise would be a $1 billion data center running CPUs, and we shrunk it down into a data center of $100 million,” Huang said. “Now, $100 million, when you put that in the cloud and shared by 100 companies, is almost nothing.”

Huang said that Nvidia’s GPUs allow startups to train models for a much lower cost than if they used a traditional computer processor.

“Now you could build something like a large language model, like a GPT, for something like $10, $20 million,” Huang said. “That’s really, really affordable.”

New competition

Nvidia isn’t the only company making GPUs for artificial intelligence uses. AMD and Intel have competing graphics processors, and big cloud companies like Google and Amazon are developing and deploying their own chips specially designed for AI workloads.

Still, “AI hardware remains strongly consolidated to NVIDIA,” according to the State of AI compute report. As of December, more than 21,000 open-source AI papers said they used Nvidia chips.

Most researchers included in the State of AI Compute Index used the V100, Nvidia’s chip that came out in 2017, but A100 grew fast in 2022 to be the third-most used Nvidia chip, just behind a $1500-or-less consumer graphics chip originally intended for gaming.

The A100 also has the distinction of being one of only a few chips to have export controls placed on it because of national defense reasons. Last fall, Nvidia said in an SEC filing that the U.S. government imposed a license requirement barring the export of the A100 and the H100 to China, Hong Kong, and Russia.

“The USG indicated that the new license requirement will address the risk that the covered products may be used in, or diverted to, a ‘military end use’ or ‘military end user’ in China and Russia,” Nvidia said in its filing. Nvidia previously said it adapted some of its chips for the Chinese market to comply with U.S. export restrictions.

The fiercest competition for the A100 may be its successor. The A100 was first introduced in 2020, an eternity ago in chip cycles. The H100, introduced in 2022, is starting to be produced in volume — in fact, Nvidia recorded more revenue from H100 chips in the quarter ending in January than the A100, it said on Wednesday, although the H100 is more expensive per unit.

The H100, Nvidia says, is the first one of its data center GPUs to be optimized for transformers, an increasingly important technique that many of the latest and top AI applications use. Nvidia said on Wednesday that it wants to make AI training over 1 million percent faster. That could mean that, eventually, AI companies wouldn’t need so many Nvidia chips.

Continue Reading

Technology

WikiLeaks whistleblower Chelsea Manning says censorship is still ‘a dominant threat’

Published

on

By

WikiLeaks whistleblower Chelsea Manning says censorship is still 'a dominant threat'

Chelsea Manning: Censorship still a dominant threat

Former U.S. Army intelligence analyst Chelsea Manning says censorship is still “a dominant threat,” advocating for a more decentralized internet to help better protect individuals online.

Her comments come amid ongoing tension linked to online safety rules, with some tech executives recently seeking to push back over content moderation concerns.

Speaking to CNBC’s Karen Tso at the Web Summit tech conference in Lisbon, Portugal, on Wednesday, Manning said that one way to ensure online privacy could be “decentralized identification,” which gives individuals the ability to control their own data.

“Censorship is a dominant threat. I think that it is a question of who’s doing the censoring, and what the purpose is — and also censorship in the 21st century is more about whether or not you’re boosted through like an algorithm, and how the fine-tuning of that seems to work,” Manning said.

“I think that social media and the monopolies of social media have sort of gotten us used to the fact that certain things that drive engagement will be attractive,” she added.

“One of the ways that we can sort of countervail that is to go back to the more decentralized and distribute the internet of the early ’90s, but make that available to more people.”

Nym Technologies Chief Security Officer Chelsea Manning at a press conference held with Nym Technologies CEO Harry Halpin in the Media Village to present NymVPN during the second day of Web Summit on November 13, 2024 in Lisbon, Portugal. 

Horacio Villalobos | Getty Images News | Getty Images

Asked how tech companies could make money in such a scenario, Manning said there would have to be “a better social contract” put in place to determine how information is shared and accessed.

“One of the things about distributed or decentralized identification is that through encryption you’re able to sort of check the box yourself, instead of having to depend on the company to provide you with a check box or an accept here, you’re making that decision from a technical perspective,” Manning said.

‘No longer secrecy versus transparency’

Manning, who works as a security consultant at Nym Technologies, a company that specializes in online privacy and security, was convicted of espionage and other charges at a court-martial in 2013 for leaking a trove of secret military files to online media publisher WikiLeaks.

She was sentenced to 35 years in prison, but was later released in 2017, when former U.S. President Barack Obama commuted her sentence.

Asked to what extent the environment has changed for whistleblowers today, Manning said, “We’re at an interesting time because information is everywhere. We have more information than ever.”

She added, “Countries and governments no longer seem to invest the same amount of time and effort in hiding information and keeping secrets. What countries seem to be doing now is they seem to be spending more time and energy spreading misinformation and disinformation.”

Manning said the challenge for whistleblowers now is to sort through the information to understand what is verifiable and authentic.

“It’s no longer secrecy versus transparency,” she added.

Continue Reading

Technology

SoftBank-backed fintech Zopa aims to double profit this year as it eyes 2025 current account launch

Published

on

By

SoftBank-backed fintech Zopa aims to double profit this year as it eyes 2025 current account launch

Jaidev Janardana, CEO of U.K. digital bank Zopa.

Zopa

LISBON, Portugal — British online lender Zopa is on track to double profits and increase annual revenue by more than a third this year amid bumper demand for its banking services, the company’s CEO told CNBC.

Zopa posted revenues of £222 million ($281.7 million) in 2023 and is expecting to cross the £300 million revenue milestone this year — that would mark a 35% annual jump.

The 2024 estimates are based on unaudited internal figures.

The firm also says it is on track to increase pre-tax profits twofold in 2024, after hitting £15.8 million last year.

Zopa, a regulated bank that is backed by Japanese giant SoftBank, has plans to venture into the world of current accounts next year as it looks to focus more on new products.

The company currently offers credit cards, personal loans and savings accounts that it offers through a mobile app — similar to other digital banks such as Monzo and Revolut which don’t operate physical branches.

“The business is doing really well. In 2024, we’ve hit or exceeded the plans across all metrics,” CEO Jaidev Janardana told CNBC in an interview Wednesday.

He said the strong performance is coming off the back of gradually improving sentiment in the U.K. economy, where Zopa operates exclusively.

Commenting on Britain’s macroeconomic conditions, Janardana said, “While it has been a rough few years, in terms of consumers, they have continued to feel the pain slightly less this year than last year.”

The market is “still tight,” he noted, adding that fintech offerings such as Zopa’s — which typically provide higher savings rates than high-street banks — become “more important” during such times.

“The proposition has become more relevant, and while it’s tight for customers, we have had to be much more constrained in terms of who we can lend to,” he said, adding that Zopa has still been able to grow despite that.

A big priority for the business going forward is product, Janardana said. The firm is developing a current account product which would allow users to spend and manage their money more easily, in a similar fashion to mainstream banking providers like HSBC and Barclays, as well as fintech upstarts such as Monzo.

What leaders are saying about AI at one of Europe's biggest tech shows

“We believe that there is more that the consumer can have in the current account space,” Janardana said. “We expect that we will launch our current account with the general public sometime next year.”

Janardana said consumers can expect a “slick” experience from Zopa’s current account offering, including the ability to view and manage multiple account bank accounts from one interface and access to competitive savings rates.

IPO ‘not top of mind’

Continue Reading

Technology

It’s ‘liquidity, stupid’: VCs say tech investing is tough amid IPO lull and ‘nuts’ AI hype

Published

on

By

It's 'liquidity, stupid': VCs say tech investing is tough amid IPO lull and 'nuts' AI hype

Edith Yeung, general partner at Race Capital, and Larry Aschebrook, founder and managing partner of G Squared, speak during a CNBC-moderated panel at Web Summit 2024 in Lisbon, Portugal.

Rita Franca | Nurphoto | Getty Images

LISBON, Portugal — It’s a tough time for the venture capital industry right now as a dearth of blockbuster initial public offerings and M&A activity has sucked liquidity from the market, while buzzy artificial intelligence startups dominate attention.

At the Web Summit tech conference in Lisbon, two venture investors — whose portfolios include the likes of multibillion-dollar AI startups Databricks Anthropic and Groq — said things have become much more difficult as they’re unable to cash out of some of their long-term bets.

“In the U.S., when you talk about the presidential election, it’s the economy stupid. And in the VC world, it’s really all about liquidity stupid,” Edith Yeung, general partner at Race Capital, an early-stage VC firm based in Silicon Valley, said in a CNBC-moderated panel earlier this week.

Liquidity is the holy grail for VCs, startup founders and early employees as it gives them a chance to realize gains — or, if things turn south, losses — on their investments.

When a VC makes an equity investment and the value of their stake increases, it’s only a gain on paper. But when a startup IPOs or sells to another company, their equity stake gets converted into hard cash — enabling them to make new investments.

Yeung said the lack of IPOs over the last couple of years had created a “really tough” environment for venture capital.

At the same, however, there’s been a rush from investors to get into buzzy AI firms.

“What’s really crazy is in the last few years, OpenAI’s domination has really been determined by Big Techs, the Microsofts of the world,” said Yeung, referring to ChatGPT-creator OpenAI’s seismic $157 billion valuation. OpenAI is backed by Microsoft, which has made a multibillion-dollar investment in the firm.

‘The IPO market is not happening’

Larry Aschebrook, founder and managing partner at late-stage VC firm G Squared, agreed that the hunt for liquidity is getting harder — even though the likes of OpenAI are seeing blockbuster funding rounds, which he called “a bit nuts.”

“You have funds and founders and employees searching for liquidity because the IPO market is not happening. And then you have funding rounds taking place of generational types of businesses,” Aschebrook said on the panel.

As important as these deals are, Aschebrook suggested they aren’t helping investors because even more money is getting tied up in illiquid, privately owned shares. G Squared itself an early backer of Anthropic, a foundational AI model startup competing with Microsoft-backed OpenAI.

Using a cooking analogy, Aschebrook suggested that venture capitalists are being starved of lucrative share sales which would lead to them realizing returns. “If you want to cook some dinner, you better sell some stock, ” he added.

Looking for opportunities beyond OpenAI

Continue Reading

Trending