Connect with us

Published

on

In an unmarked office building in Austin, Texas, two small rooms contain a handful of Amazon employees designing two types of microchips for training and accelerating generative AI. These custom chips, Inferentia and Trainium, offer AWS customers an alternative to training their large language models on Nvidia GPUs, which have been getting difficult and expensive to procure. 

“The entire world would like more chips for doing generative AI, whether that’s GPUs or whether that’s Amazon’s own chips that we’re designing,” Amazon Web Services CEO Adam Selipsky told CNBC in an interview in June. “I think that we’re in a better position than anybody else on Earth to supply the capacity that our customers collectively are going to want.”

Yet others have acted faster, and invested more, to capture business from the generative AI boom. When OpenAI launched ChatGPT in November, Microsoft gained widespread attention for hosting the viral chatbot, and investing a reported $13 billion in OpenAI. It was quick to add the generative AI models to its own products, incorporating them into Bing in February. 

That same month, Google launched its own large language model, Bard, followed by a $300 million investment in OpenAI rival Anthropic. 

It wasn’t until April that Amazon announced its own family of large language models, called Titan, along with a service called Bedrock to help developers enhance software using generative AI.

“Amazon is not used to chasing markets. Amazon is used to creating markets. And I think for the first time in a long time, they are finding themselves on the back foot and they are working to play catch up,” said Chirag Dekate, VP analyst at Gartner.

Meta also recently released its own LLM, Llama 2. The open-source ChatGPT rival is now available for people to test on Microsoft‘s Azure public cloud.

Chips as ‘true differentiation’

In the long run, Dekate said, Amazon’s custom silicon could give it an edge in generative AI. 

“I think the true differentiation is the technical capabilities that they’re bringing to bear,” he said. “Because guess what? Microsoft does not have Trainium or Inferentia,” he said.

AWS quietly started production of custom silicon back in 2013 with a piece of specialized hardware called Nitro. It’s now the highest-volume AWS chip. Amazon told CNBC there is at least one in every AWS server, with a total of more than 20 million in use. 

AWS started production of custom silicon back in 2013 with this piece of specialized hardware called Nitro. Amazon told CNBC in August that Nitro is now the highest volume AWS chip, with at least one in every AWS server and a total of more than 20 million in use.

Courtesy Amazon

In 2015, Amazon bought Israeli chip startup Annapurna Labs. Then in 2018, Amazon launched its Arm-based server chip, Graviton, a rival to x86 CPUs from giants like AMD and Intel.

“Probably high single-digit to maybe 10% of total server sales are Arm, and a good chunk of those are going to be Amazon. So on the CPU side, they’ve done quite well,” said Stacy Rasgon, senior analyst at Bernstein Research.

Also in 2018, Amazon launched its AI-focused chips. That came two years after Google announced its first Tensor Processor Unit, or TPU. Microsoft has yet to announce the Athena AI chip it’s been working on, reportedly in partnership with AMD

CNBC got a behind-the-scenes tour of Amazon’s chip lab in Austin, Texas, where Trainium and Inferentia are developed and tested. VP of product Matt Wood explained what both chips are for.

“Machine learning breaks down into these two different stages. So you train the machine learning models and then you run inference against those trained models,” Wood said. “Trainium provides about 50% improvement in terms of price performance relative to any other way of training machine learning models on AWS.”

Trainium first came on the market in 2021, following the 2019 release of Inferentia, which is now on its second generation.

Inferentia allows customers “to deliver very, very low-cost, high-throughput, low-latency, machine learning inference, which is all the predictions of when you type in a prompt into your generative AI model, that’s where all that gets processed to give you the response, ” Wood said.

For now, however, Nvidia’s GPUs are still king when it comes to training models. In July, AWS launched new AI acceleration hardware powered by Nvidia H100s. 

“Nvidia chips have a massive software ecosystem that’s been built up around them over the last like 15 years that nobody else has,” Rasgon said. “The big winner from AI right now is Nvidia.”

Amazon’s custom chips, from left to right, Inferentia, Trainium and Graviton are shown at Amazon’s Seattle headquarters on July 13, 2023.

Joseph Huerta

Leveraging cloud dominance

AWS’ cloud dominance, however, is a big differentiator for Amazon.

“Amazon does not need to win headlines. Amazon already has a really strong cloud install base. All they need to do is to figure out how to enable their existing customers to expand into value creation motions using generative AI,” Dekate said.

When choosing between Amazon, Google, and Microsoft for generative AI, there are millions of AWS customers who may be drawn to Amazon because they’re already familiar with it, running other applications and storing their data there.

“It’s a question of velocity. How quickly can these companies move to develop these generative AI applications is driven by starting first on the data they have in AWS and using compute and machine learning tools that we provide,” explained Mai-Lan Tomsen Bukovec, VP of technology at AWS.

AWS is the world’s biggest cloud computing provider, with 40% of the market share in 2022, according to technology industry researcher Gartner. Although operating income has been down year-over-year for three quarters in a row, AWS still accounted for 70% of Amazon’s overall $7.7 billion operating profit in the second quarter. AWS’ operating margins have historically been far wider than those at Google Cloud.

AWS also has a growing portfolio of developer tools focused on generative AI.

“Let’s rewind the clock even before ChatGPT. It’s not like after that happened, suddenly we hurried and came up with a plan because you can’t engineer a chip in that quick a time, let alone you can’t build a Bedrock service in a matter of 2 to 3 months,” said Swami Sivasubramanian, AWS’ VP of database, analytics and machine learning.

Bedrock gives AWS customers access to large language models made by Anthropic, Stability AI, AI21 Labs and Amazon’s own Titan.

“We don’t believe that one model is going to rule the world, and we want our customers to have the state-of-the-art models from multiple providers because they are going to pick the right tool for the right job,” Sivasubramanian said.

An Amazon employee works on custom AI chips, in a jacket branded with AWS’ chip Inferentia, at the AWS chip lab in Austin, Texas, on July 25, 2023.

Katie Tarasov

One of Amazon’s newest AI offerings is AWS HealthScribe, a service unveiled in July to help doctors draft patient visit summaries using generative AI. Amazon also has SageMaker, a machine learning hub that offers algorithms, models and more. 

Another big tool is coding companion CodeWhisperer, which Amazon said has enabled developers to complete tasks 57% faster on average. Last year, Microsoft also reported productivity boosts from its coding companion, GitHub Copilot. 

In June, AWS announced a $100 million generative AI innovation “center.” 

“We have so many customers who are saying, ‘I want to do generative AI,’ but they don’t necessarily know what that means for them in the context of their own businesses. And so we’re going to bring in solutions architects and engineers and strategists and data scientists to work with them one on one,” AWS CEO Selipsky said.

Although so far AWS has focused largely on tools instead of building a competitor to ChatGPT, a recently leaked internal email shows Amazon CEO Andy Jassy is directly overseeing a new central team building out expansive large language models, too.

In the second-quarter earnings call, Jassy said a “very significant amount” of AWS business is now driven by AI and more than 20 machine learning services it offers. Some examples of customers include Philips, 3M, Old Mutual and HSBC. 

The explosive growth in AI has come with a flurry of security concerns from companies worried that employees are putting proprietary information into the training data used by public large language models.

“I can’t tell you how many Fortune 500 companies I’ve talked to who have banned ChatGPT. So with our approach to generative AI and our Bedrock service, anything you do, any model you use through Bedrock will be in your own isolated virtual private cloud environment. It’ll be encrypted, it’ll have the same AWS access controls,” Selipsky said.

For now, Amazon is only accelerating its push into generative AI, telling CNBC that “over 100,000” customers are using machine learning on AWS today. Although that’s a small percentage of AWS’s millions of customers, analysts say that could change.

“What we are not seeing is enterprises saying, ‘Oh, wait a minute, Microsoft is so ahead in generative AI, let’s just go out and let’s switch our infrastructure strategies, migrate everything to Microsoft.’ Dekate said. “If you’re already an Amazon customer, chances are you’re likely going to explore Amazon ecosystems quite extensively.”

— CNBC’s Jordan Novet contributed to this report.

CORRECTION: This article has been updated to reflect Inferentia as the chip used for machine learning inference.

Continue Reading

Technology

CoreWeave stock slumps 14% on wider-than-expected loss ahead of lockup expiration

Published

on

By

CoreWeave stock slumps 14% on wider-than-expected loss ahead of lockup expiration

CoreWeave sinks more than 8% on quarterly results

CoreWeave‘s stock dropped 14% after the renter of artificial intelligence data centers reported a bigger-than-expected loss.

In its second quarterly financial results as a public company, CoreWeave reported an adjusted loss of 27 cents per share, compared to a 21-cent loss per share expected by analysts polled by LSEG.

CoreWeave’s results came as the lock-up period following its initial public offering is set to expire Thursday evening and potentially add volatility to shares. The term refers to a set period of time following a market debut when insiders are restricted from selling shares.

“We remain constructive long term and are encouraged by today’s data points, but see near-term upside capped by the potential CORZ related dilution and uncertainty, and the pending lock-up expiration on Thursday,” wrote analysts at Stifel, referencing the recent acquisition of Core Scientific.

Shares of Core Scientific fell 7% Wednesday.

In the current quarter, the company projects $1.26 billion to $1.30 billion in revenue. Analysts polled by LSEG forecasted $1.25 billion. CoreWeave also lifted 2025 revenue guidance to between $5.15 billion and $5.35, up from a $4.9 billion to $5.1 billion forecast provided in May and above a $5.05 billion estimate.

Read more CNBC tech news

Some analysts were hoping for stronger guidance given the stock’s massive surge since going public in March. Others highlighted light capital expenditures guidance and a delay in some spending until the fourth quarter as a potential point of weakness.

“This delay in capex highlights the uncertainty around deployment time; as go-live timing is pushed, in-period revenue recognition will be smaller,” wrote analysts at Morgan Stanley.

The AI infrastructure provider said revenue more than tripled from a year ago to $1.21 billion as it continues to benefit from surging AI demand. That also surpassed a $1.08 billion forecast from Wall Street. Finance chief Nitin Agrawal also said during a call with analysts that demand outweighs supply.

The New Jersey-based company, whose customers include OpenAI, Microsoft and Nvidia, also said it has recently signed expansion deals with hyperscale customers.

CoreWeave acquired AI model monitoring startup Weights and Biases for $1.4 billion during the period and said it finished the quarter with a $30.1 billion revenue backlog.

Stock Chart IconStock chart icon

hide content

Coreweave stock 6-month chart.

Continue Reading

Technology

How Big Tech is paying its way out of Trump’s tariffs

Published

on

By

How Big Tech is paying its way out of Trump's tariffs

Apple CEO Tim Cook (R) shakes hands with U.S. President Donald Trump during an event in the Oval Office of the White House on August 6, 2025 in Washington, DC.

Win Mcnamee | Getty Images

Top tech executives are at the forefront of a recent swathe of unprecedented deals with U.S. President Donald Trump.

In just the last few days, the White House confirmed that two U.S. chipmakers, Nvidia and Advanced Micro Devices, would be allowed to sell advanced chips to China in exchange for the U.S. government receiving a 15% cut of their revenues in the Asian country.

Apple CEO Tim Cook, meanwhile, recently announced plans to increase the firm’s U.S. investment commitment to $600 billion over the next four years. The move was widely seen as a bid to get the tech giant out of Trump’s crosshairs on tariffs — and appears to have worked for now.

Altogether, analysts say the deals show just how important it is for the world’s largest companies to find some tariff relief.

“The flurry of deal-making is an effort to secure lighter treatment from tariffs,” Paolo Pescatore, technology analyst at PP Foresight, told CNBC by email.

“In some shape or form, all of the big tech companies have been negatively impacted by tariffs. They can ill afford to fork out on millions of dollars in additional fees that will further dent profits as underlined by recent quarterly earnings,” Pescatore said.

While the devil will be in the detail of these agreements, Pescatore said that Apple leading the way with its accelerated U.S. investment will likely trigger “a domino effect” within the industry.

Apple, for its part, has long been regarded as one of the Big Tech firms most vulnerable to simmering trade tensions between the U.S. and China.

Earlier this month, Trump announced plans to impose a 100% tariff on imports of semiconductors and chips, albeit with an exemption for firms that are “building in the United States.”

Apple, which relies on hundreds of different chips for its devices and incurred $800 million in tariff costs in the June quarter, is among the firms exempt from the proposed tariffs.

A ‘hands-on’ approach

The Nvidia and AMD deal with the Trump administration has meanwhile sparked intense debate over the potential impact on the chip giants’ businesses and whether the U.S. government may seek out similar agreements with other firms.

Some strategists described the arrangement as a “shakedown,” while others suggested it may even be unconstitutional and comparing it to a tax on exports.

White House spokesperson Karoline Leavitt said Tuesday that the legality and mechanics of the 15% export tax on Nvidia and AMD were “still being ironed out.” She also hinted deals of this kind could expand to other companies in future.

Ray Wang: Having Nvidia and AMD pay 15% of China chip sales revenues to U.S. govt. is 'bizarre'

Ray Wang, founder and chairman of Constellation Research, described the Nvidia and AMD deal to pay 15% of China chip sales revenues to the U.S. government as “bizarre.”

Speaking on CNBC’s “Squawk Box” on Monday, Wang said what is “really weird” is there is still some uncertainty over whether these chips represent a national security issue.

“If the answer is no, fine OK. The government is taking a cut out of it,” Wang said. “Both Nvidia’s Jensen Huang and Lisa Su at AMD both decided that OK, we’ve got a way to get our chips into China and maybe there is something good coming out of it.”

Investor concerns

While investors initially welcomed the deal as broadly positive for both Nvidia and AMD, which once more secure access to the Chinese market, Wang said some in the industry will nevertheless be concerned.

“As an investor, you’re worried because then, is this an arbitrary decision by the government? Does every president get to play kingmaker in terms of these deals?” Wang said.

“So, I think that’s really what the concern is, and we still have additional tariffs and trade deals to come from the China negotiations,” he added.

Tech investor Dan Niles says Nvidia having access to the Chinese market is 'crucial'

Continue Reading

Technology

Amazon launches same-day delivery of meat, eggs, produce in more than 1,000 cities

Published

on

By

Amazon launches same-day delivery of meat, eggs, produce in more than 1,000 cities

An independent contractor wearing a protective mask and gloves loads Amazon Prime grocery bags into a car outside a Whole Foods Market in Berkeley, California, on Oct. 7, 2020.

David Paul Morris | Bloomberg | Getty Images

Amazon is rolling out same-day delivery of fresh foods to more pockets of the U.S. as it looks to encourage shoppers to add meat and eggs to their order while they’re browsing its sprawling online store.

The company announced Wednesday it’s bringing the service to more than 1,000 U.S. cities and towns, including Raleigh, North Carolina, Tampa, Florida, and Milwaukee, Wisconsin, with plans to reach at least 2,300 locations by the end of this year.

Amazon began testing the service in Phoenix last year and in additional cities this year, where it found shoppers frequently added strawberries, bananas, avocados and other perishables to their order.

“Many of these shoppers were first-time Amazon grocery customers who now return to shop twice as often with same-day delivery service compared to those who didn’t purchase fresh food,” the company said in a release.

Read more CNBC tech news

The service is free for Prime members on orders over $25 in most cities, or for a $2.99 fee if an order doesn’t meet that minimum. Shoppers without a Prime membership pay a $12.99 fee to use the service, regardless of order size.

Grocery delivery company Instacart‘s stock tumbled more than 11% following the announcement. Supermarket chains Kroger and Albertsons fell about 4% and 3%, respectively.

Shares of Walmart, which has been racing to compete with Amazon on speedy deliveries and offers same-day shipping for groceries, slipped 1%.

Amazon has been retooling its grocery business over the past few years.

The company has tweaked its chain of Fresh grocery stores in a bid to attract more shoppers, and it opened up fresh food delivery to shoppers who aren’t Prime members.

It’s also looked to highlight its growing business selling household staples like paper towels, cleaning supplies, bottled drinks and canned food.

In January, Amazon tapped Jason Buechel, the CEO of Whole Foods Market, the upscale grocer it acquired in 2017 for $13.7 billion, to lead its worldwide grocery stores business. Buechel announced in June that the company was bringing Whole Foods closer to the Amazon grocery umbrella as part of a reorganization.

Previously, Whole Foods had remained largely independent from Amazon’s own grocery offerings.

Continue Reading

Trending