In an unmarked office building in Austin, Texas, two small rooms contain a handful of Amazon employees designing two types of microchips for training and accelerating generative AI. These custom chips, Inferentia and Trainium, offer AWS customers an alternative to training their large language models on Nvidia GPUs, which have been getting difficult and expensive to procure.
“The entire world would like more chips for doing generative AI, whether that’s GPUs or whether that’s Amazon’s own chips that we’re designing,” Amazon Web Services CEO Adam Selipsky told CNBC in an interview in June. “I think that we’re in a better position than anybody else on Earth to supply the capacity that our customers collectively are going to want.”
Yet others have acted faster, and invested more, to capture business from the generative AI boom. When OpenAI launched ChatGPT in November, Microsoft gained widespread attention for hosting the viral chatbot, and investing a reported $13 billion in OpenAI. It was quick to add the generative AI models to its own products, incorporating them into Bing in February.
That same month, Google launched its own large language model, Bard, followed by a $300 million investment in OpenAI rival Anthropic.
It wasn’t until April that Amazon announced its own family of large language models, called Titan, along with a service called Bedrock to help developers enhance software using generative AI.
“Amazon is not used to chasing markets. Amazon is used to creating markets. And I think for the first time in a long time, they are finding themselves on the back foot and they are working to play catch up,” said Chirag Dekate, VP analyst at Gartner.
In the long run, Dekate said, Amazon’s custom silicon could give it an edge in generative AI.
“I think the true differentiation is the technical capabilities that they’re bringing to bear,” he said. “Because guess what? Microsoft does not have Trainium or Inferentia,” he said.
AWS quietly started production of custom silicon back in 2013 with a piece of specialized hardware called Nitro. It’s now the highest-volume AWS chip. Amazon told CNBC there is at least one in every AWS server, with a total of more than 20 million in use.
AWS started production of custom silicon back in 2013 with this piece of specialized hardware called Nitro. Amazon told CNBC in August that Nitro is now the highest volume AWS chip, with at least one in every AWS server and a total of more than 20 million in use.
Courtesy Amazon
In 2015, Amazon bought Israeli chip startup Annapurna Labs. Then in 2018, Amazon launched its Arm-based server chip, Graviton, a rival to x86 CPUs from giants like AMD and Intel.
“Probably high single-digit to maybe 10% of total server sales are Arm, and a good chunk of those are going to be Amazon. So on the CPU side, they’ve done quite well,” said Stacy Rasgon, senior analyst at Bernstein Research.
Also in 2018, Amazon launched its AI-focused chips. That came two years after Google announced its first Tensor Processor Unit, or TPU. Microsoft has yet to announce the Athena AI chip it’s been working on, reportedly in partnership with AMD.
CNBC got a behind-the-scenes tour of Amazon’s chip lab in Austin, Texas, where Trainium and Inferentia are developed and tested. VP of product Matt Wood explained what both chips are for.
“Machine learning breaks down into these two different stages. So you train the machine learning models and then you run inference against those trained models,” Wood said. “Trainium provides about 50% improvement in terms of price performance relative to any other way of training machine learning models on AWS.”
Trainium first came on the market in 2021, following the 2019 release of Inferentia, which is now on its second generation.
Inferentia allows customers “to deliver very, very low-cost, high-throughput, low-latency, machine learning inference, which is all the predictions of when you type in a prompt into your generative AI model, that’s where all that gets processed to give you the response, ” Wood said.
For now, however, Nvidia’s GPUs are still king when it comes to training models. In July, AWS launched new AI acceleration hardware powered by Nvidia H100s.
“Nvidia chips have a massive software ecosystem that’s been built up around them over the last like 15 years that nobody else has,” Rasgon said. “The big winner from AI right now is Nvidia.”
Amazon’s custom chips, from left to right, Inferentia, Trainium and Graviton are shown at Amazon’s Seattle headquarters on July 13, 2023.
Joseph Huerta
Leveraging cloud dominance
AWS’ cloud dominance, however, is a big differentiator for Amazon.
“Amazon does not need to win headlines. Amazon already has a really strong cloud install base. All they need to do is to figure out how to enable their existing customers to expand into value creation motions using generative AI,” Dekate said.
When choosing between Amazon, Google, and Microsoft for generative AI, there are millions of AWS customers who may be drawn to Amazon because they’re already familiar with it, running other applications and storing their data there.
“It’s a question of velocity. How quickly can these companies move to develop these generative AI applications is driven by starting first on the data they have in AWS and using compute and machine learning tools that we provide,” explained Mai-Lan Tomsen Bukovec, VP of technology at AWS.
AWS is the world’s biggest cloud computing provider, with 40% of the market share in 2022, according to technology industry researcher Gartner. Although operating income has been down year-over-year for three quarters in a row, AWS still accounted for 70% of Amazon’s overall $7.7 billion operating profit in the second quarter. AWS’ operating margins have historically been far wider than those at Google Cloud.
“Let’s rewind the clock even before ChatGPT. It’s not like after that happened, suddenly we hurried and came up with a plan because you can’t engineer a chip in that quick a time, let alone you can’t build a Bedrock service in a matter of 2 to 3 months,” said Swami Sivasubramanian, AWS’ VP of database, analytics and machine learning.
Bedrock gives AWS customers access to large language models made by Anthropic, Stability AI, AI21 Labs and Amazon’s own Titan.
“We don’t believe that one model is going to rule the world, and we want our customers to have the state-of-the-art models from multiple providers because they are going to pick the right tool for the right job,” Sivasubramanian said.
An Amazon employee works on custom AI chips, in a jacket branded with AWS’ chip Inferentia, at the AWS chip lab in Austin, Texas, on July 25, 2023.
Katie Tarasov
One of Amazon’s newest AI offerings is AWS HealthScribe, a service unveiled in July to help doctors draft patient visit summaries using generative AI. Amazon also has SageMaker, a machine learning hub that offers algorithms, models and more.
Another big tool is coding companion CodeWhisperer, which Amazon said has enabled developers to complete tasks 57% faster on average. Last year, Microsoft also reported productivity boosts from its coding companion, GitHub Copilot.
“We have so many customers who are saying, ‘I want to do generative AI,’ but they don’t necessarily know what that means for them in the context of their own businesses. And so we’re going to bring in solutions architects and engineers and strategists and data scientists to work with them one on one,” AWS CEO Selipsky said.
Although so far AWS has focused largely on tools instead of building a competitor to ChatGPT, a recently leaked internal email shows Amazon CEO Andy Jassy is directly overseeing a new central team building out expansive large language models, too.
In the second-quarter earnings call, Jassy said a “very significant amount” of AWS business is now driven by AI and more than 20 machine learning services it offers. Some examples of customers include Philips, 3M, Old Mutual and HSBC.
The explosive growth in AI has come with a flurry of security concerns from companies worried that employees are putting proprietary information into the training data used by public large language models.
“I can’t tell you how many Fortune 500 companies I’ve talked to who have banned ChatGPT. So with our approach to generative AI and our Bedrock service, anything you do, any model you use through Bedrock will be in your own isolated virtual private cloud environment. It’ll be encrypted, it’ll have the same AWS access controls,” Selipsky said.
For now, Amazon is only accelerating its push into generative AI, telling CNBC that “over 100,000” customers are using machine learning on AWS today. Although that’s a small percentage of AWS’s millions of customers, analysts say that could change.
“What we are not seeing is enterprises saying, ‘Oh, wait a minute, Microsoft is so ahead in generative AI, let’s just go out and let’s switch our infrastructure strategies, migrate everything to Microsoft.’ Dekate said. “If you’re already an Amazon customer, chances are you’re likely going to explore Amazon ecosystems quite extensively.”
— CNBC’s Jordan Novet contributed to this report.
CORRECTION: This article has been updated to reflect Inferentia as the chip used for machine learning inference.
The stock was on pace for its best day since March 2020.
The Wall Street Journal reported that Broadcom may consider a play for the company’s chip design and marketing segment, citing people familiar with the matter, while TSMC is interested in a stake or complete control of Intel’s factories. The companies have not filed bids and talks are largely informal, the Journal reported.
The iconic American chipmaker’s stock has continued to sink lower in recent years, shedding billions in market value. Intel fell behind on the artificial intelligence tailwinds that have swept up the broader semiconductor sector.
Elon Musk leaves after a meeting with Indian Prime Minister Narendra Modi at Blair House, in Washington, D.C., U.S., February 13, 2025.
Nathan Howard | Reuters
A law firm that represents Tesla and Elon Musk has written proposed legislation that would alter Delaware corporate law, according to a person directly familiar with the drafting of the bill.
The proposed legislation, drafted by Richards, Layton & Finger, or RLF, would amend Delaware General Corporation Law, and if adopted, could pave the way for the reinstatement of Musk’s 2018 CEO pay package at Tesla, worth tens of billions in options.
RLF confirmed their involvement to CNBC.
“Statutory changes are necessary to restore the core principles that have been the hallmark of Delaware for over a century and ensure that Delaware remains the preeminent jurisdiction for incorporation,” Lisa Schmidt, president of RLF, said in a statement.
The bill was introduced to the Delaware General Assembly on Monday and would require approval by the state’s two chambers as well as Gov. Matt Meyer before becoming a law.
The pay package Tesla granted to Musk in 2018 was the largest CEO compensation plan in public corporate history, but the it was ordered to be rescinded last year by the Delaware Court of Chancery.
In her ruling, Chancellor Kathaleen McCormick wrote that the pay plan was inappropriately set by Tesla’s board, which was controlled by Musk, and that it was approved by shareholders who were misled by Tesla’s proxy materials before they were asked to vote on it.
Under the proposed legislation, Musk might no longer be considered a “controller” of Tesla, said Brian JM Quinn, Boston College Law professor. Transactions that involve self-dealing with controllers or directors would be subject to less review than they are now, Quinn said. Those transactions range from going-private deals, to mergers and acquisitions, and board and executive compensation decisions.
“The real role of corporate law is to protect minority investors,” Quinn said. “With this bill, the legislature is saying ‘Now you know what? Protect them less.'”
The proposed legislation would also limit the documents that minority stakeholders are able to obtain through “books and records” inspection requests,Quinn said. Those stakeholders would be limited to formal items like a certificate of incorporation or minutes of stockholder meetings but they’d lose access to informal communications like emails or other messages between board members and executives, Quinn said.
After the Court of Chancery’s ruling last year, Musk started a campaign against companies incorporating in Delaware and moved the site of incorporation for his businesses out of the state. He has aimed his ire at Chancellor McCormick with repeated and disparaging posts about her on X, his social network.
Other prominent executives, including Coinbase CEO Brian Armstrong and Bill Ackman of Pershing Square, have also voiced criticism of the Delaware judiciary.
“Delaware has taken some heat for supposedly being too hard on controller transactions,” said Renee Zaytsev, partner at Boies Schiller and co-chair of the firm’s securities and shareholder dispute practice.
“These amendments seem to be a course correction that would make it significantly easier for boards and controllers to avoid judicial scrutiny of their transactions,” she said.
Tesla and Musk did not respond to requests for comment.
Elon Musk’s xAI on Tuesday unveiled its latest artificial intelligence model, Grok 3, claiming it can outperform offerings from OpenAI and China’s DeepSeek based on early testing, which included standardized tests on math, science and coding.
“We’re very excited to present Grok 3, which is, we think, an order of magnitude more capable than Grok 2 in a very short period of time,” Musk said at a demonstration of Grok 3 that was streamed on his social media platform X.
The team also said it was launching a new product called “Deep Search,” which would act as a “next generation search engine.”
Grok 3 will be rolled out for premium X subscribers later in the day, and will also be accessible through a separate subscription for the model’s web and app versions, the xAI team said.
Speaking at The World Governments Summit in Dubai last week Musk had dubbed the model “scary smart,” with powerful reasoning capabilities, claiming it outperformed all other existing models in xAI’s internal tests.
“This might be the last time that an AI is better than Grok,” Musk said at the time, adding that it was trained on “a lot of synthetic data,” and was capable of reflecting upon its mistakes to achieve logical consistency.
The xAI team claimed that an early iteration of Grok 3 had been given better ratings than existing competitors on Chatbot Arena, a crowdsourced website that pits different AI models against each other in blind tests.
Toward the end of the product demo, Musk said that the company will keep improving the model.
“We should emphasize that this is kind of a beta, meaning that you should expect some imperfections at first, but we will improve it rapidly, almost every day,” he said, adding that the voice assistance for the model would be released at a later time.
Intense competition
Musk, who has been quite vocal about the potential dangers of artificial intelligence, started xAI in 2023 entering the generative AI market that includes OpenAI’s ChatGPT.
In September last year, OpenAI launched its most advanced model, the o1, which came with reasoning abilities and was able to solve relatively complex science, coding and math tasks.
Musk, along with Sam Altman, helped create OpenAI as a nonprofit in 2015.
However, in recent years Musk and OpenAI’s leadership have been feuding. Musk recently led an investor group that submitted a proposal to buy the AI startup’s nonprofit parent for $97.4 billion — an offer OpenAI declined.
Last month, Chinese start-up DeepSeek shocked the AI market when it released a technical paper that claimed one of its open source models was able to rival the performance of OpenAI’s o1 model despite using a cheaper, less energy-intensive process.
It accomplished the feat in the face of the U.S. restricting leading AI chipmaker Nvidia from selling its cutting-edge GPUs — used for training AI models — to China.
XAI has a “Colossus supercomputer,” for training AI, which it said last year was utilizing a cluster of 100,000 advanced Nvidia GPUs for AI training. On Tuesday, the company revealed that it doubled the size of its GPU cluster for the training of Grok 3.
While many AI and tech experts have told CNBC that DeepSeek has intensified AI competition, showing what can be done with less advanced technology, others are more skeptical about its impact.