Inside a sprawling lab at Google headquarters in Mountain View, California, hundreds of server racks hum across several aisles, performing tasks far less ubiquitous than running the world’s dominant search engine or executing workloads for Google Cloud’s millions of customers.
Instead, they’re running tests on Google’s own microchips, called Tensor Processing Units, or TPUs.
Originally trained for internal workloads, Google’s TPUs have been available to cloud customers since 2018. In July, Apple revealed it uses TPUs to train AI models underpinning Apple Intelligence. Google also relies on TPUs to train and run its Gemini chatbot.
“The world sort of has this fundamental belief that all AI, large language models, are being trained on Nvidia, and of course Nvidia has the lion’s share of training volume. But Google took its own path here,” said Futurum Group CEO Daniel Newman. He’s been covering Google’s custom cloud chips since they launched in 2015.
Google was the first cloud provider to make custom AI chips. Three years later, Amazon Web Services announced its first cloud AI chip, Inferentia. Microsoft‘s first custom AI chip, Maia, wasn’t announced until the end of 2023.
But being first in AI chips hasn’t translated to a top spot in the overall rat race of generative AI. Google’s faced criticism for botched product releases, and Gemini came out more than a year after OpenAI’s ChatGPT.
Google Cloud, however, has gained momentum due in part to AI offerings. Google parent company Alphabet reported cloud revenue rose 29% in the most recent quarter, surpassing $10 billion in quarterly revenues for the first time.
“The AI cloud era has completely reordered the way companies are seen, and this silicon differentiation, the TPU itself, may be one of the biggest reasons that Google went from the third cloud to being seen truly on parity, and in some eyes, maybe even ahead of the other two clouds for its AI prowess,” Newman said.
‘A simple but powerful thought experiment’
In July, CNBC got the first on-camera tour of Google’s chip lab and sat down with the head of custom cloud chips, Amin Vahdat. He was already at Google when it first toyed with the idea of making chips in 2014.
Amin Vahdat, VP of Machine Learning, Systems and Cloud AI at Google, holds up TPU Version 4 at Google headquarters in Mountain View, California, on July 23, 2024.
Marc Ganley
“It all started with a simple but powerful thought experiment,” Vahdat said. “A number of leads at the company asked the question: What would happen if Google users wanted to interact with Google via voice for just 30 seconds a day? And how much compute power would we need to support our users?”
“We realized that we could build custom hardware, not general purpose hardware, but custom hardware — Tensor Processing Units in this case — to support that much, much more efficiently. In fact, a factor of 100 more efficiently than it would have been otherwise,” Vahdat said.
Google data centers still rely on general-purpose central processing units, or CPUs, and Nvidia’s graphics processing units, or GPUs. Google’s TPUs are a different type of chip called an application-specific integrated circuit, or ASIC, which are custom-built for specific purposes. The TPU is focused on AI. Google makes another ASIC focused on video called a Video Coding Unit.
The TPU, however, is what set Google apart. It was the first of its kind when it launched in 2015. Google TPUs still dominate among custom cloud AI accelerators, with 58% of the market share, according to The Futurum Group.
Google coined the term based on the algebraic term “tensor,” referring to the large-scale matrix multiplications that happen rapidly for advanced AI applications.
With the second TPU release in 2018, Google expanded the focus from inference to training and made them available for its cloud customers to run workloads, alongside market-leading chips such as Nvidia’s GPUs.
“If you’re using GPUs, they’re more programmable, they’re more flexible. But they’ve been in tight supply,” said Stacy Rasgon, senior analyst covering semiconductors at Bernstein Research.
The AI boom has sent Nvidia’s stock through the roof, catapulting the chipmaker to a $3 trillion market cap in June, surpassing Alphabet and jockeying with Apple and Microsoft for position as the world’s most valuable public company.
“Being candid, these specialty AI accelerators aren’t nearly as flexible or as powerful as Nvidia’s platform, and that is what the market is also waiting to see: Can anyone play in that space?” Newman said.
Now that we know Apple’s using Google’s TPUs to train its AI models, the real test will come as those full AI features roll out on iPhones and Macs next year.
Broadcom and TSMC
It’s no small feat to develop alternatives to Nvidia’s AI engines. Google’s sixth generation TPU, called Trillium, is set to come out later this year.
Google showed CNBC the sixth version of its TPU, Trillium, in Mountain View, California, on July 23, 2024. Trillium is set to come out later in 2024.
Marc Ganley
“It’s expensive. You need a lot of scale,” Rasgon said. “And so it’s not something that everybody can do. But these hyperscalers, they’ve got the scale and the money and the resources to go down that path.”
The process is so complex and costly that even the hyperscalers can’t do it alone. Since the first TPU, Google’s partnered with Broadcom, a chip developer that also helps Meta design its AI chips. Broadcom says it’s spent more than $3 billion to make these partnerships happen.
“AI chips — they’re very complex. There’s lots of things on there. So Google brings the compute,” Rasgon said. “Broadcom does all the peripheral stuff. They do the I/O and the SerDes, all of the different pieces that go around that compute. They also do the packaging.”
Then the final design is sent off for manufacturing at a fabrication plant, or fab — primarily those owned by the world’s largest chipmaker, Taiwan Semiconductor Manufacturing Company, which makes 92% of the world’s most advanced semiconductors.
When asked if Google has any safeguards in place should the worst happen in the geopolitical sphere between China and Taiwan, Vahdat said, “It’s certainly something that we prepare for and we think about as well, but we’re hopeful that actually it’s not something that we’re going to have to trigger.”
Protecting against those risks is the primary reason the White House is handing out $52 billion in CHIPS Act funding to companies building fabs in the U.S. — with the biggest portions going to Intel, TSMC, and Samsung to date.
Processors and power
Google showed CNBC its new Axion CPU,
Marc Ganley
“Now we’re able to bring in that last piece of the puzzle, the CPU,” Vahdat said. “And so a lot of our internal services, whether it’s BigQuery, whether it’s Spanner, YouTube advertising and more are running on Axion.”
Google is late to the CPU game. Amazon launched its Graviton processor in 2018. Alibaba launched its server chip in 2021. Microsoft announced its CPU in November.
When asked why Google didn’t make a CPU sooner, Vahdat said, “Our focus has been on where we can deliver the most value for our customers, and there it has been starting with the TPU, our video coding units, our networking. We really thought that the time was now.”
All these processors from non-chipmakers, including Google’s, are made possible by Arm chip architecture — a more customizable, power-efficient alternative that’s gaining traction over the traditional x86 model from Intel and AMD. Power efficiency is crucial because, by 2027, AI servers are projected to use up as much power every year as a country like Argentina. Google’s latest environmental report showed emissions rose nearly 50% from 2019 to 2023 partly due to data center growth for powering AI.
“Without having the efficiency of these chips, the numbers could have wound up in a very different place,” Vahdat said. “We remain committed to actually driving these numbers in terms of carbon emissions from our infrastructure, 24/7, driving it toward zero.”
It takes a massive amount of water to cool the servers that train and run AI. That’s why Google’s third-generation TPU started using direct-to-chip cooling, which uses far less water. That’s also how Nvidia’s cooling its latest Blackwell GPUs.
Despite challenges, from geopolitics to power and water, Google is committed to its generative AI tools and making its own chips.
“I’ve never seen anything like this and no sign of it slowing down quite yet,” Vahdat said. “And hardware is going to play a really important part there.”
A robotic machine manufactures a semiconductor chip at a stall to show investors during The Advantage Assam 2.0 Investment Summit in Guwahati, India, on Feb. 25, 2025.
Nurphoto | Nurphoto | Getty Images
India wants to become a global chip major, but the odds are steep: competition is fierce, and India is a late entrant in the race to make the most advanced chips.
In 2022, when the U.S. restricted exports of its advanced AI chips to China to curb Beijing’s access to cutting-edge technology, a global race for semiconductor self-reliance began.
For India, it offered an opportunity: the country wants to reduce dependence on imports, secure chips for strategic sectors, and capture a bigger share of the global electronics market shifting away from China.
India is one of the world’s largest consumers of electronics, but it has no local chip industry and plays a minimal role in the global supply chain. New Delhi’s “Semiconductor Mission” aims to change that.
The ambition is bold. It wants to create a full supply chain — from design to fabrication, testing and packaging — on Indian soil.
As of this month, the country has approved 10 semiconductor projects with total investment of 1.6 trillion rupees ($18.2 billion). These include two semiconductor fabrication plants, and multiple testing and packing factories.
India also has a pool of engineering talent that is already employed by global chip design companies.
Yet progress so far has been uneven, and neither the investments nor talent pool is enough to make India’s chip ambitions a reality, say experts.
“India needs more than a few fabs or ATP facilities (i.e., more than a few “shiny objects.”) It needs a dynamic and deep and long-term ecosystem,” said Stephen Ezell, vice president for global innovation policy at the Information Technology and Innovation Foundation, a science and technology policy think tank.
Ezell says that leading semiconductor manufacturers consider “as many as 500 discrete factors” before they set up multi-billion-dollar fab investments. These include talent, tax, trade, technology policies, labor rates and laws and customs policies — all areas where India has work to do.
New Delhi’s policy push
In May, the Indian government added a new element to its chip ambition: a scheme to support electronic component manufacturing, addressing a critical bottleneck.
Until now, chipmakers had no local demand for their product as there are hardly any electronic component manufacturing companies, such as phone camera companies, in India.
Researchers inside the semiconductor fabrication lab at the Centre for Nano Science and Engineering, at the Indian Institute of Science, in Bangalore.
Manjunath Kiran | Afp | Getty Images
But the new policy offers financial support to companies producing active and passive electronic components, creating a potential domestic buyer-supplier base that chip manufacturers can plug into.
In 2022, the country also pivoted from its strategy of providing superior incentives to fabrication units making chips of 28nm or less. When it comes to chips, the smaller the size, the higher the performance with improved energy efficiency. These chips can be used in new technologies like advanced AI and quantum computing by packing more transistors into the same space.
But this approach wasn’t helping India develop its nascent semiconductor industry, so New Delhi now covers 50% of the project costs of all fabrication units, regardless of chip size, and of chip testing and packing units.
Fab companies from Taiwan and the U.K., and semiconductor packaging companies from the U.S. and South Korea have all shown interest in aiding India’s semiconductor ambitions.
“The Indian government has doled out generous incentives to attract semiconductor manufacturers to India,” said Ezell, but he stressed that “those sorts of investments aren’t sustainable forever.”
The long road
The biggest chip project in India currently is the 910-billion-rupee ($11 billion) semiconductor fabrication plant being built in Prime Minister Narendra Modi’s home state of Gujarat by Tata Electronics, in partnership with Taiwan’s Powerchip Semiconductor Manufacturing Corp.
The unit will make chips for power management integrated circuits, display drivers, microcontrollers and high-performance computing logic, Tata Electronics said, which can be used in AI, automotive, computing and data storage industries.
The U.K.’s Clas-SiC Wafer Fab has also tied up with India’s SiCSem to set up the country’s first commercial compound fab in the eastern state of Odisha.
These compound semiconductors can be used in missiles, defence equipment, electric vehicles, consumer appliances and solar power inverters, according to a government press release.
“The coming 3-4 years is pivotal for advancing India’s semiconductor goals,” said Sujay Shetty, managing director of semiconductor at PwC India.
Establishing operational silicon fabrication facilities and overcoming technical and infrastructural hurdles that extend beyond incentives will be a key milestone, according to Shetty.
Opportunities beyond fab
Fabrication sites need to meet stringent requirements, such as being in areas free from floods and vibrations, with reliable road connectivity — which can present ongoing logistical considerations for some regions.
India also needs specialty chemical suppliers that meet “ultra-high purity standards essential for advanced semiconductor manufacturing,” Shetty added.
Beyond chip fabrication plants, many medium-sized companies in India have shown interest in setting up chip testing and packaging units. Several Indian groups are also entering the segment, attracted by its higher margins and lower capital intensity compared to fabs.
“Outsourced semiconductor assembly and testing (OSAT) represents a significant opportunity for India, though clarifying market access and demand channels will be important for sustained growth,” says Shetty.
Success in this area will see India enter the global chip industry, but New Delhi is still a long way from locally developing and manufacturing the cutting-edge of chip technology: 2nm semiconductors.
These 2nm chips offer better performance and power efficiency due to their smaller transistor size. Taiwan Semiconductor Manufacturing Corporation will start mass-producing cutting-edge 2nm chips later this year, according to an FT report.
NEW DELHI, INDIA – MAY 14: Union Minister of Railways, Information and Broadcasting, Electronics and Information Technology Ashwini Vaishnaw briefing the media on Cabinet decisions at National Media Centre on May 14, 2025 in New Delhi, India.
Hindustan Times | Hindustan Times | Getty Images
Last week, Indian minister Ashwini Vaishnaw, who was in Bengaluru to inaugurate a new office of semiconductor design firm ARM, said the British company will design the “most advanced chips used in AI servers, drones, mobile phone chips of 2 nm” from the south Indian city.
But experts say the role of local talent is likely to be limited to non-core design testing and validation, as the core intellectual property for chip designs is often held in locations like the U.S. or Singapore, where established IP regimes support such activities.
“India has sufficient talent in design space, because unlike semiconductor manufacturing and testing that has come up in the last 2 years, design has been there since 1990s,” said Jayanth BR, a recruiter with over 15 years of experience in hiring for global semiconductor companies in India.
He said global companies usually outsource “block-level” design validation work to India.
Going beyond this is something India’s government will need to solve if it wants to fulfil its semiconductor ambitions.
“India may consider updating its IP laws to address new forms of IP, like digital content and software. Of course, improving enforcement mechanisms will go a long way in protecting IP rights,” says Sajai Singh, a partner at Mumbai-based JSA Advocates & Solicitors.
“Our competition is with countries like the U.S., Europe, and Taiwan, which not only have strong IP laws, but also a more established ecosystem for chip design.”
Nvidia CEO Jensen Huang attends the “Winning the AI Race” Summit in Washington D.C., U.S., July 23, 2025.
Kent Nishimura | Reuters
Nvidia CEO Jensen Huang and OpenAI CEO Sam Altman on Monday commented on President Donald Trump’s decision to increase the cost of hiring overseas workers on visas.
Trump on Friday announced that he would raise the fee for an H-1B visa to $100,000, leaving companies scrambling. Employers now must have documentation of the payment prior to filing an H-1B petition on behalf of a worker. Applicants will have their petitions restricted for 12 months until the payment is made, according to the White House.
Huang and Altman responded to the changes in an interview with CNBC’s Jon Fortt, where the two executives announced that Nvidia will invest $100 billion in OpenAI as the artificial intelligence lab sets out to build hundreds of billions of dollars-worth of data centers based around the chipmaker’s AI processors.
“We want all the brightest minds to come to the U.S. and remember immigration is the foundation of the American Dream,” Huang said Monday. “We represent the American Dream. And so I think immigration is really important to our company and is really important to our nation’s future, and I’m glad to see President Trump making the moves he’s making.”
OpenAI CEO Sam Altman also expressed a positive outlook on Trump’s changes.
“We need to get the smartest people in the country, and streamlining that process and also sort of outlining financial incentives seems good to me,” Altman said.
The new $100,000 fee would be a seismic shift for U.S. technology and finance sectors, which rely on the H-1B program for highly skilled immigrants, particularly from India and China. Those two countries accounted for 71% and 11.7% of visa holders last year, respectively.
Those who already have H-1B visas and are located outside the U.S. will not be required to pay the fee in order to re-enter. Many employers use H-1B workers to fill the gaps in these highly technical roles that are not found within the American labor supply.
— CNBC tech reporter Annie Palmer contributed to this report.
President Donald Trump speaks before signing executive orders in the Oval Office at the White House on September 19, 2025 in Washington, DC.
Andrew Harnik | Getty Images
President Donald Trump raised the fee for an H-1B visa to $100,000 on Friday, leaving companies scrambling to respond.
With many left wondering whether their careers will remain in tact, here’s a breakdown of the new H-1B fees:
What did Trump change?
As of Sunday, H-1B visa applications will require a $100,000 payment. Previously, visa fees ranged from $2,000 to $5,000 per application, depending on the size of the company.
Employers now must have documentation of the payment prior to filing an H-1B petition on behalf of a worker. Applicants will have their petitions restricted for 12 months until the payment is made, according to the White House.
Who does this impact?
The fee will only be applied to new H-1B applicants, not renewals or current visa holders, according to White House press secretary Karoline Leavitt. The fee will be implemented in the upcoming lottery cycle.
Those who already have H-1B visas and are located outside the U.S. will not be required to pay the fee in order to re-enter.
Leavitt also clarified that the $100,000 is a one-time payment and not an annual charge.
Exceptions can be made to any immigrant whose employment is deemed essential in the national interest by the Secretary of Homeland Security and does not pose a threat to the security or welfare of the U.S.
Employees with B visas who have start dates prior to October 2026 will also receive additional guidance in order to prevent using those temporary business visas as a workaround for H-1B visas.
Who are these workers and why are they needed?
H-1B visas allows highly skilled foreign professionals to work in specialty occupations that generally require at least a bachelor’s degree to fulfill the role. Jobs in the fields of science, technology, engineering and math, or STEM, usually qualify.
Many employers use H-1B workers to fill the gaps in these highly technical roles that are not found within the American labor supply.
Companies in the tech and finance sectors rely heavily on these specially-skilled immigrants, particularly from India and China, which accounted for 71% and 11.7% of visa holders last year, respectively.
How many H-1B visas does the tech industry use every year?
The current annual cap for H-1B visas is 65,000, along with an additional 20,000 visas for foreign professionals with a master’s degree or doctorate from a U.S. institution. A lottery system is used to select additional petitions if demand exceeds the cap.
Since 2012, about 60% or more of approved H-1B workers had computer-related jobs, according to Pew Research.
Amazon was the top employer for H-1B holders in the fiscal year 2025, sponsoring over 10,000 applicants by the end of June, according to U.S. Citizenship and Immigration Services. Microsoft and Meta had over 5,000 each, while Apple and Google rounded out the top six with over 4,000 approvals.
