Google issues apology, incident report for hourslong cloud outage
Thomas Kurian, CEO of Google Cloud, speaks at a cloud computing conference held by the company in 2019.
Michael Short | Bloomberg | Getty Images
Google apologized for a major outage that the company said was caused by multiple layers of flawed recent updates.
The company released an incident report late on Friday that explained hours of downtime on Thursday. More than 70 Google cloud services stopped working properly across the globe, knocking down or disrupting dozens of third-party services, including Cloudflare, OpenAI and Shopify. Gmail, Google Calendar, Google Drive, Google Meet and other first-party products also malfunctioned.
“We deeply apologize for the impact this outage has had,” Google wrote in the incident report. “Google Cloud customers and their users trust their businesses to Google, and we will do better. We apologize for the impact this has had not only on our customers’ businesses and their users but also on the trust of our systems. We are committed to making improvements to help avoid outages like this moving forward.”
Thomas Kurian, CEO of Google’s cloud unit, also posted about the outage in an X post on Thursday, saying “we regret the disruption this caused our customers.”
Google in May added a new feature to its “quota policy checks” for evaluating automated incoming requests, but the new feature wasn’t immediately tested in real-world situations, the company wrote in the incident report. As a result, the company’s systems didn’t know how to properly handle data from the new feature, which included blank entries. Those blank entries were then sent out to all Google Cloud data center regions, which prompted the crashes, the company wrote.
Engineers figured out the issue in 10 minutes, according to the company. However, the entire incident went on for seven hours after that, with the crash leading to an overload in some larger regions.
As it released the feature, Google did not use feature flags, an increasingly common industry practice that allows for slow implementation to minimize impact if problems occur. Feature flags would have caught the issue before the feature became widely available, Google said.
Going forward, Google will change its architecture so if one system fails, it can still operate without crashing, the company said. Google said it will also audit all systems and improve its communications “both automated and human, so our customers get the information they need asap to react to issues.”
— CNBC’s Jordan Novet contributed to this report.
WATCH: Google buyouts highlight tech’s cost-cutting amid AI CapEx boom
An Amazon worker moves boxes on Amazon Prime Day in the East Village of New York City, July 11, 2023.
Spencer Platt | Getty Images
Amazon is extending its Prime Day discount bonanza, announcing that the annual sale will run four days this year.
The 96-hour event will start at 12:01 a.m. PT on July 8, and continue through July 11, Amazon said in a release.
For the first time, the company will roll out themed “deal drops” that change daily and are available “while supplies last.” Amazon has in recent years toyed with adding more limited-run and invite-only deals during Prime Day events to create a feeling of urgency or scarcity.
Amazon launched Prime Day in 2015 as a way to secure new members for its $139-a-year loyalty program, and to promote its own products and services while providing a sales boost in the middle of the year. In 2019, the company made Prime Day a 48-hour event, and it’s since added a second Prime Day-like event in the fall.
Prime Day is also a significant revenue driver for other retailers, which often host competing discount events.
Technology
SK Hynix shares extend gains to over 2-decade highs as parent group reportedly plans AI data center
OpenAI CEO Sam Altman speaks during the Snowflake Summit in San Francisco on June 2, 2025.
Justin Sullivan | Getty Images News | Getty Images
OpenAI has been awarded a $200 million contract to provide the U.S. Defense Department with artificial intelligence tools.
The department announced the one-year contract on Monday, months after OpenAI said it would collaborate with defense technology startup Anduril to deploy advanced AI systems for “national security missions.”
“Under this award, the performer will develop prototype frontier AI capabilities to address critical national security challenges in both warfighting and enterprise domains,” the Defense Department said. It’s the first contract with OpenAI listed on the Department of Defense’s website.
Anduril received a $100 million defense contract in December. Weeks earlier, OpenAI rival Anthropic said it would work with Palantir and Amazon to supply its AI models to U.S. defense and intelligence agencies.
Sam Altman, OpenAI’s co-founder and CEO, said in a discussion with OpenAI board member and former National Security Agency leader Paul Nakasone at a Vanderbilt University event in April that “we have to and are proud to and really want to engage in national security areas.”
OpenAI did not immediately respond to a request for comment.
The Defense Department specified that the contract is with OpenAI Public Sector LLC, and that the work will mostly occur in the National Capital Region, which encompasses Washington, D.C., and several nearby counties in Maryland and Virginia.
Meanwhile, OpenAI is working to build additional computing power in the U.S. In January, Altman appeared alongside President Donald Trump at the White House to announce the $500 billion Stargate project to build AI infrastructure in the U.S.
The new contract will represent a small portion of revenue at OpenAI, which is generating over $10 billion in annualized sales. In March, the company announced a $40 billion financing round at a $300 billion valuation.
In April, Microsoft, which supplies cloud infrastructure to OpenAI, said the U.S. Defense Information Systems Agency has authorized the use of the Azure OpenAI service with secret classified information.
-
Sports3 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoStory injured on diving stop, exits Red Sox game
-
Sports2 years agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Sports2 years agoButton battles heat exhaustion in NASCAR debut
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike