Inside a sprawling lab at Google headquarters in Mountain View, California, hundreds of server racks hum across several aisles, performing tasks far less ubiquitous than running the world’s dominant search engine or executing workloads for Google Cloud’s millions of customers.
Instead, they’re running tests on Google’s own microchips, called Tensor Processing Units, or TPUs.
Originally trained for internal workloads, Google’s TPUs have been available to cloud customers since 2018. In July, Apple revealed it uses TPUs to train AI models underpinning Apple Intelligence. Google also relies on TPUs to train and run its Gemini chatbot.
“The world sort of has this fundamental belief that all AI, large language models, are being trained on Nvidia, and of course Nvidia has the lion’s share of training volume. But Google took its own path here,” said Futurum Group CEO Daniel Newman. He’s been covering Google’s custom cloud chips since they launched in 2015.
Google was the first cloud provider to make custom AI chips. Three years later, Amazon Web Services announced its first cloud AI chip, Inferentia. Microsoft‘s first custom AI chip, Maia, wasn’t announced until the end of 2023.
But being first in AI chips hasn’t translated to a top spot in the overall rat race of generative AI. Google’s faced criticism for botched product releases, and Gemini came out more than a year after OpenAI’s ChatGPT.
Google Cloud, however, has gained momentum due in part to AI offerings. Google parent company Alphabet reported cloud revenue rose 29% in the most recent quarter, surpassing $10 billion in quarterly revenues for the first time.
“The AI cloud era has completely reordered the way companies are seen, and this silicon differentiation, the TPU itself, may be one of the biggest reasons that Google went from the third cloud to being seen truly on parity, and in some eyes, maybe even ahead of the other two clouds for its AI prowess,” Newman said.
‘A simple but powerful thought experiment’
In July, CNBC got the first on-camera tour of Google’s chip lab and sat down with the head of custom cloud chips, Amin Vahdat. He was already at Google when it first toyed with the idea of making chips in 2014.
Amin Vahdat, VP of Machine Learning, Systems and Cloud AI at Google, holds up TPU Version 4 at Google headquarters in Mountain View, California, on July 23, 2024.
Marc Ganley
“It all started with a simple but powerful thought experiment,” Vahdat said. “A number of leads at the company asked the question: What would happen if Google users wanted to interact with Google via voice for just 30 seconds a day? And how much compute power would we need to support our users?”
“We realized that we could build custom hardware, not general purpose hardware, but custom hardware — Tensor Processing Units in this case — to support that much, much more efficiently. In fact, a factor of 100 more efficiently than it would have been otherwise,” Vahdat said.
Google data centers still rely on general-purpose central processing units, or CPUs, and Nvidia’s graphics processing units, or GPUs. Google’s TPUs are a different type of chip called an application-specific integrated circuit, or ASIC, which are custom-built for specific purposes. The TPU is focused on AI. Google makes another ASIC focused on video called a Video Coding Unit.
The TPU, however, is what set Google apart. It was the first of its kind when it launched in 2015. Google TPUs still dominate among custom cloud AI accelerators, with 58% of the market share, according to The Futurum Group.
Google coined the term based on the algebraic term “tensor,” referring to the large-scale matrix multiplications that happen rapidly for advanced AI applications.
With the second TPU release in 2018, Google expanded the focus from inference to training and made them available for its cloud customers to run workloads, alongside market-leading chips such as Nvidia’s GPUs.
“If you’re using GPUs, they’re more programmable, they’re more flexible. But they’ve been in tight supply,” said Stacy Rasgon, senior analyst covering semiconductors at Bernstein Research.
The AI boom has sent Nvidia’s stock through the roof, catapulting the chipmaker to a $3 trillion market cap in June, surpassing Alphabet and jockeying with Apple and Microsoft for position as the world’s most valuable public company.
“Being candid, these specialty AI accelerators aren’t nearly as flexible or as powerful as Nvidia’s platform, and that is what the market is also waiting to see: Can anyone play in that space?” Newman said.
Now that we know Apple’s using Google’s TPUs to train its AI models, the real test will come as those full AI features roll out on iPhones and Macs next year.
Broadcom and TSMC
It’s no small feat to develop alternatives to Nvidia’s AI engines. Google’s sixth generation TPU, called Trillium, is set to come out later this year.
Google showed CNBC the sixth version of its TPU, Trillium, in Mountain View, California, on July 23, 2024. Trillium is set to come out later in 2024.
Marc Ganley
“It’s expensive. You need a lot of scale,” Rasgon said. “And so it’s not something that everybody can do. But these hyperscalers, they’ve got the scale and the money and the resources to go down that path.”
The process is so complex and costly that even the hyperscalers can’t do it alone. Since the first TPU, Google’s partnered with Broadcom, a chip developer that also helps Meta design its AI chips. Broadcom says it’s spent more than $3 billion to make these partnerships happen.
“AI chips — they’re very complex. There’s lots of things on there. So Google brings the compute,” Rasgon said. “Broadcom does all the peripheral stuff. They do the I/O and the SerDes, all of the different pieces that go around that compute. They also do the packaging.”
Then the final design is sent off for manufacturing at a fabrication plant, or fab — primarily those owned by the world’s largest chipmaker, Taiwan Semiconductor Manufacturing Company, which makes 92% of the world’s most advanced semiconductors.
When asked if Google has any safeguards in place should the worst happen in the geopolitical sphere between China and Taiwan, Vahdat said, “It’s certainly something that we prepare for and we think about as well, but we’re hopeful that actually it’s not something that we’re going to have to trigger.”
Protecting against those risks is the primary reason the White House is handing out $52 billion in CHIPS Act funding to companies building fabs in the U.S. — with the biggest portions going to Intel, TSMC, and Samsung to date.
Processors and power
Google showed CNBC its new Axion CPU,
Marc Ganley
“Now we’re able to bring in that last piece of the puzzle, the CPU,” Vahdat said. “And so a lot of our internal services, whether it’s BigQuery, whether it’s Spanner, YouTube advertising and more are running on Axion.”
Google is late to the CPU game. Amazon launched its Graviton processor in 2018. Alibaba launched its server chip in 2021. Microsoft announced its CPU in November.
When asked why Google didn’t make a CPU sooner, Vahdat said, “Our focus has been on where we can deliver the most value for our customers, and there it has been starting with the TPU, our video coding units, our networking. We really thought that the time was now.”
All these processors from non-chipmakers, including Google’s, are made possible by Arm chip architecture — a more customizable, power-efficient alternative that’s gaining traction over the traditional x86 model from Intel and AMD. Power efficiency is crucial because, by 2027, AI servers are projected to use up as much power every year as a country like Argentina. Google’s latest environmental report showed emissions rose nearly 50% from 2019 to 2023 partly due to data center growth for powering AI.
“Without having the efficiency of these chips, the numbers could have wound up in a very different place,” Vahdat said. “We remain committed to actually driving these numbers in terms of carbon emissions from our infrastructure, 24/7, driving it toward zero.”
It takes a massive amount of water to cool the servers that train and run AI. That’s why Google’s third-generation TPU started using direct-to-chip cooling, which uses far less water. That’s also how Nvidia’s cooling its latest Blackwell GPUs.
Despite challenges, from geopolitics to power and water, Google is committed to its generative AI tools and making its own chips.
“I’ve never seen anything like this and no sign of it slowing down quite yet,” Vahdat said. “And hardware is going to play a really important part there.”
A soldier walks next to a Tesla Cybertruck, which was donated to the National Guard, after powerful winds fueling devastating wildfires in the Los Angeles area forced people to evacuate, in the Pacific Palisades neighborhood on the west side of Los Angeles, California, U.S. Jan. 13, 2025.
Daniel Cole | Reuters
Tesla started offering discounts on new Cybertruck vehicles in its inventory this week, according to listings on the company’s website.
Discounts are as high as $1,600 off new Cybertrucks, with the reduced price depending on configuration, and up to around $2,600 for demo versions of the trucks in inventory, the listings show. Production of the angular, unpainted steel pickups has reportedly slowed in recent weeks at Tesla’s factory in Austin, Texas.
Deliveries of the unconventional pickup began reaching customers in 2023. CEO Elon Musk originally unveiled the Cybertruck in 2019 and said it would cost around $40,000, but its base price in the U.S. was closer to $80,000 over the course of 2024.
Wall Street previously viewed the Cybertruck as an important driver of growth for Tesla’s core automotive sales.
While the Cybertruck outsold the Ford Lightning F-150 last year in the U.S. and became the fifth best-selling EV domestically, according to data tracked by Cox Automotive, its high price, repeat recalls and production issues in Austin hampered growth. In November, Tesla initiated its sixth recall in a year to replace defective drive inverters.
As CNBC previously reported, Tesla’s deliveries declined slightly year-over-year in 2024, even as EV demand worldwide reached a record. A slew of new competitive models from a wide range of automakers eroded Tesla’s market share.
According to Cox data, full-year EV sales reached an estimated 1.3 million in 2024 in the U.S., an increase of 7.3% from the prior year. But Tesla’s sales for the year declined by about 37,000 vehicles.
The Tesla Model Y SUV and Model 3 sedan ranked as the top two best-selling EVs by a wide margin. But both older, more affordable Tesla models saw sales drop from the previous year. Cox estimated Tesla sold around 38,965 Cybertrucks in the U.S. last year.
In recent days, Musk apologized to customers in California for delays in delivering their Cybertrucks. He said the trucks are now being used to bring supplies and wireless internet service to people in Los Angeles impacted by devastating wildfires.
“Apologies to those expecting Cybertruck deliveries in California over the next few days,” Musk wrote on X. “We need to use those trucks as mobile base stations to provide power to Starlink Internet terminals in areas of LA without connectivity. A new truck will be delivered end of week.”
David Solomon, CEO of Goldman Sachs, speaks during the Reuters NEXT conference, in New York City, U.S., December 10, 2024.
Mike Segar | Reuters
Goldman Sachs CEO David Solomon says there’s an end in sight to the multi-year IPO drought.
“It’s going to pick up,” Solomon said on Wednesday, in an on-stage interview with Cisco CEO Chuck Robbins at a summit hosted by the computer networking company in Silicon Valley. “It’s been slow, it’s been turned off.”
Solomon, who flew to California for the event just after his Wall Street bank reported fourth-quarter results that blew past analysts’ estimates, said the capital markets broadly are showing signs of life ahead of President-elect Donald Trump’s inauguration next week.
The tech IPO market has largely been dormant since the end of 2021, when tech stocks started falling out of favor due to soaring inflation and rising interest rates. Mergers and acquisitions have been difficult in technology because of hefty regulation that’s restricted the ability for the biggest companies to grow through dealmaking.
Solomon said the mood is changing, and he expects momentum M&A as well as in IPOs.
“We have a more constructive kind of optimism, which always helps,” Solomon said. He later added that, “broadly speaking, I think it’s an improved business environment.”
Earlier in the day, Solomon said on his company’s earnings call that Trump’s election and a swing back to Republican power in Washington is already starting to make an impact in the business world. He noted on the call that “there is a significant backlog from sponsors and an overall increased appetite for dealmaking supported by an improved regulatory backdrop.”
Solomon’s comments on the call and at the Cisco event came on a day when the S&P 500 posted his biggest gain since November, helped by a tame inflation report and Goldman’s results. Goldman’s stock popped 6% on Wednesday.
While the stock market has had a strong two-year run and the S&P 500 and Nasdaq hit fresh records last month, IPOs have yet to see a resurgence. Cloud software vendor ServiceTitandebuted on the Nasdaq in December, marking the first significant venture-backed IPO in the U.S. since Rubrik in April.
“The values came down after 2021, people are growing back into those values,” Solomon said at the Cisco summit.
Some companies have said they’re ready. Chipmaker Cerebras filed to go public in September, but the process was slowed down due to a review by the Treasury Department’s Committee on Foreign Investment in the U.S., or CFIUS. In November, online lender Klarna said it had confidentially filed IPO paperwork with the SEC.
Though he’s bullish about what’s coming, Solomon said that there are structural reasons not to go public. He said 25 years ago there were roughly 13,000 public companies in the U.S., and today that number has come down to 3,800. There are higher standards around disclosure for being public, and there’s now tons of private capital available “at scale.”
“It’s not fun being a public company,” Solomon acknowledged. “Who would want to be a public company?”
If TikTok does indeed go dark on Sunday for Americans, there may be a tool for them to continue accessing the popular social app: VPNs.
The Chinese-owned app is set to be removed from mobile app stores and the web for U.S. users on Sunday as a result of a law signed by President Joe Biden in April 2024 requiring that the app be sold to a qualified buyer before the deadline.
Barring a last-minute sale or reprieve from the Supreme Court, the app will almost certainly vanish from the app stores for iPhones and Android phones. It won’t be removed from people’s phones, but the app could stop working.
TikTok plans to shut its service for Americans on Sunday, meaning that even those who already have the app downloaded won’t be able to continue using it, according to reports this week from Reuters and The Information. Apple and Google didn’t comment on their plans for taking down the apps from their app stores on Sunday.
“Basically, an app or a website can check where users came from,” said Justas Palekas, a head of product at IProyal.com, a proxy service. “Based on that, then they can impose restrictions based on their location.”
Masking your physical internet access point
That may stop most users, but for the particularly driven Americans, using VPNs might allow them to continue using the app.
VPNs and a related business-to-business technology called proxies work by tunneling a user’s internet traffic through a server in another country, making it look like they are accessing the internet from a location different than the one they are physically in.
This works because every time a computer connects to the internet, it is identified through an IP number, which is a 12-digit number that is different for every single computer. The first six digits of the number identifies the network, which also includes information about the physical region the request came from.
In China, people have used VPNs for years to get around the country’s firewall, which blocks U.S. websites such as Google and Facebook. VPNs saw big spikes in traffic when India banned TikTok in 2020, and people often use VPNs to watch sporting events from countries where official broadcasts aren’t available.
As of 2022, the VPN market was worth nearly $38 billion, according to the VPN Trust Initiative, a lobbying group.
“We consistently see significant spikes in VPN demand when access to online platforms is restricted, and this situation is no different,” said Lauren Hendry Parsons, privacy advocate at ExpressVPN, a VPN provider that costs $5 per month to use.
“We’re not here to endorse TikTok, but the looming U.S. ban highlights why VPNs matter— millions rely on them for secure, private, and unrestricted access to the internet,” ProtonVPN posted on social media earlier this week. ProtonVPN offers its service for $10 a month.
The price of VPNs
Both ExpressVPN and ProtonVPN allow users to set their internet-access location.
Most VPN services charge a monthly fee to pay for their servers and traffic, but some use a business model where they collect user data or traffic trends, such as when Meta offered a free VPN so it could keep an eye on which competitors’ apps were growing quickly.
A key tradeoff for those who use VPN is speed due to requests having to flow through a middleman computer to mask a users’ physical location.
And although VPNs have worked in the past when governments have banned apps, that doesn’t ensure that VPNs will work if TikTok goes dark. It won’t be clear if ExpressVPN would be able to access TikTok until after the ban takes place, Parsons told CNBC in an email. It’s also possible that TikTok may be able to determine Americans who try to use VPNs to access the app.
(L-R) Sarah Baus of Charleston, S.C., holds a sign that reads “Keep TikTok” as she and other content creators Sallye Miley of Jackson, Mississippi, and Callie Goodwin of Columbia, S.C., stand outside the U.S. Supreme Court Building as the court hears oral arguments on whether to overturn or delay a law that could lead to a ban of TikTok in the U.S., on January 10, 2025 in Washington, DC.
Andrew Harnik | Getty Images
VPNs and proxies to evade regional restrictions have been part of the internet’s landscape for decades, but their use is increasing as governments seek to ban certain services or apps.
Apps are removed by government request all the time. Nearly 1500 apps were removed in regions due to government takedown demands in 2023, according to Apple, with over 1,000 of them in China. Most of them are fringe apps that break laws such as those against gambling, or Chinese video game rules, but increasingly, countries are banning apps for national security or economic development reasons.
Now, the U.S. is poised to ban one of the most popular apps in the country — with 115 million users, it was the second most downloaded app of 2024 across both iOS and Android, according to an estimate provided to CNBC from Sensor Tower, a market intelligence firm.
“As we witness increasing attempts to fragment and censor the internet, the role of VPNs in upholding internet freedom is becoming increasingly critical,” Parsons said.