Connect with us

Published

on

In an unmarked office building in Austin, Texas, two small rooms contain a handful of Amazon employees designing two types of microchips for training and accelerating generative AI. These custom chips, Inferentia and Trainium, offer AWS customers an alternative to training their large language models on Nvidia GPUs, which have been getting difficult and expensive to procure. 

“The entire world would like more chips for doing generative AI, whether that’s GPUs or whether that’s Amazon’s own chips that we’re designing,” Amazon Web Services CEO Adam Selipsky told CNBC in an interview in June. “I think that we’re in a better position than anybody else on Earth to supply the capacity that our customers collectively are going to want.”

Yet others have acted faster, and invested more, to capture business from the generative AI boom. When OpenAI launched ChatGPT in November, Microsoft gained widespread attention for hosting the viral chatbot, and investing a reported $13 billion in OpenAI. It was quick to add the generative AI models to its own products, incorporating them into Bing in February. 

That same month, Google launched its own large language model, Bard, followed by a $300 million investment in OpenAI rival Anthropic. 

It wasn’t until April that Amazon announced its own family of large language models, called Titan, along with a service called Bedrock to help developers enhance software using generative AI.

“Amazon is not used to chasing markets. Amazon is used to creating markets. And I think for the first time in a long time, they are finding themselves on the back foot and they are working to play catch up,” said Chirag Dekate, VP analyst at Gartner.

Meta also recently released its own LLM, Llama 2. The open-source ChatGPT rival is now available for people to test on Microsoft‘s Azure public cloud.

Chips as ‘true differentiation’

In the long run, Dekate said, Amazon’s custom silicon could give it an edge in generative AI. 

“I think the true differentiation is the technical capabilities that they’re bringing to bear,” he said. “Because guess what? Microsoft does not have Trainium or Inferentia,” he said.

AWS quietly started production of custom silicon back in 2013 with a piece of specialized hardware called Nitro. It’s now the highest-volume AWS chip. Amazon told CNBC there is at least one in every AWS server, with a total of more than 20 million in use. 

AWS started production of custom silicon back in 2013 with this piece of specialized hardware called Nitro. Amazon told CNBC in August that Nitro is now the highest volume AWS chip, with at least one in every AWS server and a total of more than 20 million in use.

Courtesy Amazon

In 2015, Amazon bought Israeli chip startup Annapurna Labs. Then in 2018, Amazon launched its Arm-based server chip, Graviton, a rival to x86 CPUs from giants like AMD and Intel.

“Probably high single-digit to maybe 10% of total server sales are Arm, and a good chunk of those are going to be Amazon. So on the CPU side, they’ve done quite well,” said Stacy Rasgon, senior analyst at Bernstein Research.

Also in 2018, Amazon launched its AI-focused chips. That came two years after Google announced its first Tensor Processor Unit, or TPU. Microsoft has yet to announce the Athena AI chip it’s been working on, reportedly in partnership with AMD

CNBC got a behind-the-scenes tour of Amazon’s chip lab in Austin, Texas, where Trainium and Inferentia are developed and tested. VP of product Matt Wood explained what both chips are for.

“Machine learning breaks down into these two different stages. So you train the machine learning models and then you run inference against those trained models,” Wood said. “Trainium provides about 50% improvement in terms of price performance relative to any other way of training machine learning models on AWS.”

Trainium first came on the market in 2021, following the 2019 release of Inferentia, which is now on its second generation.

Inferentia allows customers “to deliver very, very low-cost, high-throughput, low-latency, machine learning inference, which is all the predictions of when you type in a prompt into your generative AI model, that’s where all that gets processed to give you the response, ” Wood said.

For now, however, Nvidia’s GPUs are still king when it comes to training models. In July, AWS launched new AI acceleration hardware powered by Nvidia H100s. 

“Nvidia chips have a massive software ecosystem that’s been built up around them over the last like 15 years that nobody else has,” Rasgon said. “The big winner from AI right now is Nvidia.”

Amazon’s custom chips, from left to right, Inferentia, Trainium and Graviton are shown at Amazon’s Seattle headquarters on July 13, 2023.

Joseph Huerta

Leveraging cloud dominance

AWS’ cloud dominance, however, is a big differentiator for Amazon.

“Amazon does not need to win headlines. Amazon already has a really strong cloud install base. All they need to do is to figure out how to enable their existing customers to expand into value creation motions using generative AI,” Dekate said.

When choosing between Amazon, Google, and Microsoft for generative AI, there are millions of AWS customers who may be drawn to Amazon because they’re already familiar with it, running other applications and storing their data there.

“It’s a question of velocity. How quickly can these companies move to develop these generative AI applications is driven by starting first on the data they have in AWS and using compute and machine learning tools that we provide,” explained Mai-Lan Tomsen Bukovec, VP of technology at AWS.

AWS is the world’s biggest cloud computing provider, with 40% of the market share in 2022, according to technology industry researcher Gartner. Although operating income has been down year-over-year for three quarters in a row, AWS still accounted for 70% of Amazon’s overall $7.7 billion operating profit in the second quarter. AWS’ operating margins have historically been far wider than those at Google Cloud.

AWS also has a growing portfolio of developer tools focused on generative AI.

“Let’s rewind the clock even before ChatGPT. It’s not like after that happened, suddenly we hurried and came up with a plan because you can’t engineer a chip in that quick a time, let alone you can’t build a Bedrock service in a matter of 2 to 3 months,” said Swami Sivasubramanian, AWS’ VP of database, analytics and machine learning.

Bedrock gives AWS customers access to large language models made by Anthropic, Stability AI, AI21 Labs and Amazon’s own Titan.

“We don’t believe that one model is going to rule the world, and we want our customers to have the state-of-the-art models from multiple providers because they are going to pick the right tool for the right job,” Sivasubramanian said.

An Amazon employee works on custom AI chips, in a jacket branded with AWS’ chip Inferentia, at the AWS chip lab in Austin, Texas, on July 25, 2023.

Katie Tarasov

One of Amazon’s newest AI offerings is AWS HealthScribe, a service unveiled in July to help doctors draft patient visit summaries using generative AI. Amazon also has SageMaker, a machine learning hub that offers algorithms, models and more. 

Another big tool is coding companion CodeWhisperer, which Amazon said has enabled developers to complete tasks 57% faster on average. Last year, Microsoft also reported productivity boosts from its coding companion, GitHub Copilot. 

In June, AWS announced a $100 million generative AI innovation “center.” 

“We have so many customers who are saying, ‘I want to do generative AI,’ but they don’t necessarily know what that means for them in the context of their own businesses. And so we’re going to bring in solutions architects and engineers and strategists and data scientists to work with them one on one,” AWS CEO Selipsky said.

Although so far AWS has focused largely on tools instead of building a competitor to ChatGPT, a recently leaked internal email shows Amazon CEO Andy Jassy is directly overseeing a new central team building out expansive large language models, too.

In the second-quarter earnings call, Jassy said a “very significant amount” of AWS business is now driven by AI and more than 20 machine learning services it offers. Some examples of customers include Philips, 3M, Old Mutual and HSBC. 

The explosive growth in AI has come with a flurry of security concerns from companies worried that employees are putting proprietary information into the training data used by public large language models.

“I can’t tell you how many Fortune 500 companies I’ve talked to who have banned ChatGPT. So with our approach to generative AI and our Bedrock service, anything you do, any model you use through Bedrock will be in your own isolated virtual private cloud environment. It’ll be encrypted, it’ll have the same AWS access controls,” Selipsky said.

For now, Amazon is only accelerating its push into generative AI, telling CNBC that “over 100,000” customers are using machine learning on AWS today. Although that’s a small percentage of AWS’s millions of customers, analysts say that could change.

“What we are not seeing is enterprises saying, ‘Oh, wait a minute, Microsoft is so ahead in generative AI, let’s just go out and let’s switch our infrastructure strategies, migrate everything to Microsoft.’ Dekate said. “If you’re already an Amazon customer, chances are you’re likely going to explore Amazon ecosystems quite extensively.”

— CNBC’s Jordan Novet contributed to this report.

CORRECTION: This article has been updated to reflect Inferentia as the chip used for machine learning inference.

Continue Reading

Technology

Amazon to gain creative control of James Bond franchise from Broccoli family

Published

on

By

Amazon to gain creative control of James Bond franchise from Broccoli family

Daniel Craig stars as James Bond in “No Time To Die.”

Source: MGM

Amazon is set to take creative control over the lucrative James Bond movie franchise from the Broccoli family, the company announced Thursday.

The James Bond films have long been produced by Michael Wilson and Barbara Broccoli, who inherited the control from their father Albert “Cubby” Broccoli. Wilson and Broccoli will now give creative control to MGM Studios, which Amazon acquired for $8.45 billion in 2021.

Amazon gained distribution rights to the Bond franchise after the MGM acquisition, but not creative control.

As part of the deal, Amazon’s MGM Studios, Wilson and Broccoli formed a new joint venture to house the Bond intellectual property rights, and they will remain co-owners of the franchise.

“We are grateful to the late Albert R. Broccoli and Harry Saltzman for bringing James Bond to movie theatres around the world, and to Michael G. Wilson and Barbara Broccoli for their unyielding dedication and their role in continuing the legacy of the franchise that is cherished by legions of fans worldwide,” said Mike Hopkins, Amazon’s head of Prime Video and MGM Studios, in a statement. “We are honored to continue this treasured heritage, and look forward to ushering in the next phase of the legendary 007 for audiences around the world.”

Wilson and Broccoli said in a release that they are both stepping back from producing the Bond films to focus on other projects.

“Barbara and I agree, it is time for our trusted partner, Amazon MGM Studios, to lead James Bond into the future,” Wilson said.

In a nod to the deal, Amazon founder and Executive Chairman Jeff Bezos wrote in a post on X, “Who’d you pick as the next Bond?”

The Bond film franchise, which spans more than 60 years, is one of the highest-grossing series in history.

The valuable IP stands to be a boon for Amazon’s sprawling media and entertainment business, which includes the Prime Video streaming service. Prime Video is one of the key perks of Amazon Prime, the company’s mainstay subscription service that costs $139 a year. As of 2021, the company said it had more than 200 million Prime subscribers worldwide.

Don’t miss these insights from CNBC PRO

Continue Reading

Technology

OpenAI tops 400 million users despite DeepSeek’s emergence

Published

on

By

OpenAI tops 400 million users despite DeepSeek's emergence

Brad Lightcap of OpenAI.

Courtesy: OpenAI

OpenAI appears to be growing quickly despite increasing competition. 

The San Francisco-based tech company had 400 million weekly active users as of February, up 33% from 300 million in December, the company’s chief operating officer, Brad Lightcap, told CNBC. These numbers have not been previously reported.

Lightcap pointed to the “natural progression” of ChatGPT as it becomes more useful and familiar to a broader group of people.

“People hear about it through word of mouth. They see the utility of it. They see their friends using it,” Lightcap said in an interview, adding that it takes time for individuals to find use cases that resonate. “There’s an overall effect of people really wanting these tools, and seeing that these tools are really valuable.”

OpenAI is seeing that spill over to its growing enterprise business. The company now has 2 million paying enterprise users, roughly doubling from September, said Lightcap, pointing out that often employees will use ChatGPT personally and suggest to their companies that they implement the tool. 

“We get a lot of benefits, and a tail wind from the organic consumer adoption where people already have familiarity with the product,” he said. “There’s really healthy growth, on a different curve.”

Developer traffic has also doubled in the past six months, quintupling for the company’s “reasoning” model o3, according to Lightcap. Developers use OpenAI to integrate the technology into their own applications. OpenAI counts Uber, Morgan Stanley, Moderna and T-Mobile among some of its largest enterprise customers.

Lightcap likened this usage to cloud services, which Amazon Web Services pioneered two decades ago. While the consumer business may grow faster since people can adopt it at will, enterprise is in the “process of building up,” he said. 

“There’s a buying cycle there, and a learning process that goes into scaling an enterprise business,” Lightcap said. “AI is going to be like cloud services. It’s going to be something that you can’t run a business that ultimately is not really running on these very powerful models underneath the surface.”

The DeepSeek effect

OpenAI’s growth comes amid new competition from Chinese competitor DeepSeek, which roiled tech markets in January as investors feared it would hamper future profitability of U.S. artificial intelligence companies and their dominance. Megacap tech companies were hit especially hard. Nvidia lost 17% on the Monday DeepSeek made waves, wiping off almost $600 billion in market value.

Later that week, OpenAI accused DeepSeek of improperly harvesting its models in a technique known as distillation. Lightcap said the new competition hasn’t changed the way OpenAI thinks about open source, their product road map or mega-spending plans.

“DeepSeek is a testament to how much AI is like entered the public consciousness in the mainstream — it would have been unfathomable two years ago,” he said. “It’s a moment that shows how powerful these models are and how much people really care.”

Besides DeepSeek’s emergence, OpenAI has also been dealing with a tense time on the legal front. 

Billionaire Elon Musk, a company co-founder, has sued OpenAI for breach of contract as it attempts to convert into a for-profit. Microsoft has poured billions into the company while SoftBank is close to finalizing a $40 billion investment that could value the company at close to $300 billion, according to sources familiar with the deal.

Musk and a group of investors bid to buy the nonprofit’s assets for $97.4 billion earlier this month. In a letter to Musk’s attorney, OpenAI’s lawyer said the company’s board determined that Musk’s “much-publicized ‘bid’ is in fact not a bid at all.” OpenAI Chairman Bret Taylor said in a statement that the company “is not for sale.”

“The numbers tell the story,” Lightcap said. “We try to be very transparent about where we stand on all of this. (Musk) is a competitor. He’s competing. It’s an unorthodox way of competing.”

Don’t miss these insights from CNBC PRO

DeepSeek is causing reckoning for OpenAI, Anthropic, says Redpoint Ventures' Logan Bartlett

Continue Reading

Technology

Chinese smartphone firm Oppo launches slim $1,870 folding phone to rival Samsung, Huawei

Published

on

By

Chinese smartphone firm Oppo launches slim ,870 folding phone to rival Samsung, Huawei

Oppo’s new Find N5 folding smartphone on display at a press briefing in London.

Ryan Browne | CNBC

Chinese smartphone firm Oppo has unveiled its new flagship folding phone Thursday, touting a slimmer body and artificial intelligence-focused features in a bid to compete with high-end foldable devices from the likes of Samsung and Huawei.

The company’s Find N5 phone that can fold in half, will retail at a starting price of 2,499 Singapore dollars ($1,867.70).

When it’s folded shut, the Find N5 looks like a normal bar-shaped phone with a 6.62-inch display. The device can then be folded outward to show a larger, 8.12-inch tablet.

Most notably, the phone has an ultra-thin design. When closed, it measures 8.93 millimeters thick, while when opened out in tablet form, the Find N5 has a depth of 4.21 millimeters.

That’s slimmer than Samsung’s Galaxy Fold 6, which the South Korean tech giant released last year.

Inside the device is a razer-thin 5,600 milliampere-hour (mAh) battery that’s no bigger than a credit card. Oppo said the battery incorporates a silicon-carbon material, which enables high battery capacity despite its small size.

Oppo is hoping it can win business from the likes of Samsung and Chinese tech giant Huawei, both major smartphone players seeking to shake the market out of an innovation slowdown with flashy new models that can bend.

'Sea of sameness': Are smartphone makers out of ideas?

AI assistant features

Like many other smartphone makers, Oppo is investing more into artificial intelligence-focused features on the device.

The Oppo Find N5 has a triple-camera setup that includes a telephoto lens that can zoom in up to 30x thanks to assistance from an AI-powered image enhancement feature, dubbed AI Telescope Zoom.

It also comes with a personal AI assistant that can interpret and summarize documents, generate summaries of phone calls and translate video calls and other content displayed on the screen.

Addressing concerns around privacy, Oppo said that some data is processed directly on the device while other information is stored in the cloud. In international markets, Oppo is using Google as its AI and cloud computing technology partner.

Controlling a Mac with an Android phone

Oppo also talked up a new feature that enables users to connect their phone to a Mac computer. Using an app called O+ Connect, users can link the Find N5 to any Mac desktop machine and instantly transfer photos and other files between devices — so long as they’re connected to the same Wi-Fi network.

Users can also choose to remotely control a Mac from the Find N5. The Mac’s display can shut off and then reappear on the Find N5’s screen. The remote control feature only requires mobile internet or Wi-Fi to sync up a Mac device’s data with the Find N5 in real-time.

Huawei launches $3,660 trifold phone outside of China for the first time

Continue Reading

Trending