OpenAI and Anthropic agree to let U.S. AI Safety Institute test and evaluate new models
Jakub Porzycki | Nurphoto | Getty Images
OpenAI and Anthropic, the two most richly valued artificial intelligence startups, have agreed to let the U.S. AI Safety Institute test their new models before releasing them to the public, following increased concerns in the industry about safety and ethics in AI.
The institute, housed within the Department of Commerce at the National Institute of Standards and Technology (NIST), said in a press release that it will get “access to major new models from each company prior to and following their public release.”
The group was established after the Biden-Harris administration issued the U.S. government’s first-ever executive order on artificial intelligence in October 2023, requiring new safety assessments, equity and civil rights guidance and research on AI’s impact on the labor market.
“We are happy to have reached an agreement with the US AI Safety Institute for pre-release testing of our future models,” OpenAI CEO Sam Altman wrote in a post on X. OpenAI also confirmed to CNBC on Thursday that, in the past year, the company has doubled its number of weekly active users from late last year to 200 million. Axios was first to report on the number.
The news comes a day after reports surfaced that OpenAI is in talks to raise a funding round valuing the company at more than $100 billion. Thrive Capital is leading the round and will invest $1 billion, according to a source with knowledge of the matter who asked not to be named because the details are confidential.
Anthropic, founded by ex-OpenAI research executives and employees, was most recently valued at $18.4 billion. Anthropic counts Amazon as a leading investor, while OpenAI is heavily backed by Microsoft.
The agreements between the government, OpenAI and Anthropic “will enable collaborative research on how to evaluate capabilities and safety risks, as well as methods to mitigate those risks,” according to Thursday’s release.
Jason Kwon, OpenAI’s chief strategy officer, told CNBC in a statement that, “We strongly support the U.S. AI Safety Institute’s mission and look forward to working together to inform safety best practices and standards for AI models.”
Jack Clark, co-founder of Anthropic, said the company’s “collaboration with the U.S. AI Safety Institute leverages their wide expertise to rigorously test our models before widespread deployment” and “strengthens our ability to identify and mitigate risks, advancing responsible AI development.”
A number of AI developers and researchers have expressed concerns about safety and ethics in the increasingly for-profit AI industry. Current and former OpenAI employees published an open letter on June 4, describing potential problems with the rapid advancements taking place in AI and a lack of oversight and whistleblower protections.
“AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this,” they wrote. AI companies, they added, “currently have only weak obligations to share some of this information with governments, and none with civil society,” and they can not be “relied upon to share it voluntarily.”
Days after the letter was published, a source familiar to the mater confirmed to CNBC that the FTC and the Department of Justice were set to open antitrust investigations into OpenAI, Microsoft and Nvidia. FTC Chair Lina Khan has described her agency’s action as a “market inquiry into the investments and partnerships being formed between AI developers and major cloud service providers.”
On Wednesday, California lawmakers passed a hot-button AI safety bill, sending it to Governor Gavin Newsom’s desk. Newsom, a Democrat, will decide to either veto the legislation or sign it into law by Sept. 30. The bill, which would make safety testing and other safeguards mandatory for AI models of a certain cost or computing power, has been contested by some tech companies for its potential to slow innovation.
WATCH: Google, OpenAI and others oppose California AI safety bill
The Nasdaq MarketSite in New York, June 9, 2023.
Michael Nagle | Bloomberg | Getty Images
Silicon Valley executives and financiers publicly opened their wallets in support of President Donald Trump’s 2024 presidential run. The early returns in 2025 aren’t great, to say the least.
Following Trump’s sweeping tariff plan announced Wednesday, the Nasdaq suffered steep consecutive daily drops to finish 10% lower for the week, the index’s worst performance since the beginning of the Covid pandemic in 2020.
The tech industry’s leading CEO’s rushed to contribute to Trump’s inauguration in January and paraded to Washington, D.C., for the event. Since then, it’s been a slog.
The market can always turn around, but economists and investors aren’t optimistic, and concerns are building of a potential recession. The seven most valuable U.S. tech companies lost a combined $1.8 trillion in market cap in two days.
Apple slid 14% for the week, its biggest drop in more than five years. Tesla, led by top Trump adviser Elon Musk, plunged 9.2% and is now down more than 40% for the year. Musk contributed close to $300 million to help propel Trump back to the White House.
Nvidia, Meta and Amazon all suffered double-digit drops for the week. For Amazon, a ninth straight weekly decline marks its longest such losing streak since 2008.
With Wall Street selling out of risky assets on concern that widespread tariff hikes will punish the U.S. and global economy, the fallout has drifted down to the IPO market. Online lender Klarna and ticketing marketplace StubHub delayed their IPOs due to market turbulence, just weeks after filing with the Securities and Exchange Commission, and fintech company Chime is also reportedly delaying its listing.
CoreWeave, a provider of artificial intelligence infrastructure, last week became the first venture-backed company to raise more than $1 billion in a U.S. IPO since 2021. But the company slashed its offering, and trading has been very volatile in its opening days on the market. The stock plunged 12% on Friday, leaving it 17% above its offer price but below the bottom of its initial range.
“You couldn’t create a worse market and macro environment to go public,” said Phil Haslett, co-founder of EquityZen, a platform for investing in private companies. “Way too much turbulence. All flights are grounded until further notice.”
CoreWeave investor Mark Klein of SuRo Capital previously told CNBC that the company could be the first in an “IPO parade.” Now he’s backtracking.
“It appears that the IPO parade has been temporarily halted,” Klein told CNBC by email on Friday. “The current tariff situation has prompted these companies to pause and assess its impact.”
‘Cave rapidly’
During last year’s presidential campaign, prominent venture capitalists like Marc Andreessen backed Trump, expecting that his administration would usher in a boom and eliminate some of the hurdles to startup growth set up by the Biden administration. Andreessen and his partner, Ben Horowitz, said in July that their financial support of the Trump campaign was due to what they called a better “little tech agenda.”
A spokesperson for Andreessen Horowitz declined to comment.
Some techies who supported Trump in the campaign have taken to social media to defend their positions.
Venture capitalist Keith Rabois, a managing director at Khosla Ventures, posted on X on Thursday that “Trump Derangement Syndrome has morphed into Tariff Derangement Syndrome.” He said tariffs aren’t inflationary, are effective at reducing fentanyl imports, and he expects that “most other countries will cave and cave rapidly.”
That was before China’s Finance Ministry said on Friday that it will impose a 34% tariff on all goods imported from the U.S. starting on April 10.
At Sequoia Capital, which is the biggest investor in Klarna, outspoken Trump supporter Shaun Maguire, wrote on X, “The first long-term thinking President of my lifetime,” and said in a separate post that, “The price of stocks says almost nothing about the long term health of an economy.”
However, Allianz Chief Economic Advisor Mohamed El-Erian warned on Friday that Trump’s extensive raft of import tariffs are putting the U.S. economy at risk of recession.
“You’ve had a major repricing of growth prospects, with a recession in the U.S. going up to 50% probability, you’ve seen an increase in inflation expectations, up to 3.5%,” he told CNBC’s Silvia Amaro on the sidelines of the Ambrosetti Forum in Cernobbio, Italy.
Former Microsoft CEOs Bill Gates, left, and Steve Ballmer, center, pose for photos with CEO Satya Nadella during an event celebrating the 50th Anniversary of Microsoft on April 4, 2025 in Redmond, Washington.Â
Stephen Brashear | Getty Images
Meanwhile, executives at tech’s megacap companies were largely silent this week, and their public relations representatives declined to provide comments about their thinking.
Microsoft CEO Satya Nadella was in the awkward position on Friday of celebrating his company’s 50th anniversary at corporate headquarters in Redmond, Washington. Alongside Microsoft’s prior two CEOs, Bill Gates and Steve Ballmer, Nadella sat down with CNBC’s Andrew Ross Sorkin for a televised interview that was planned well before Trump’s tariff announcement.
When asked about the tariffs at the top of the interview, Nadella effectively dodged the question and avoided expressing his views about whether the new policies will hamper Microsoft’s business.
Ballmer, who was succeeded by Nadella in 2014, acknowledged to Sorkin that “disruption is very hard on people” and that, “as a Microsoft shareholder, this kind of thing is not good.” Ballmer and Gates are two of the 12 wealthiest people in the world thanks to their Microsoft fortunes.
C-suites may not be able to stay quiet for long, especially if the recent turmoil spills into next week.
Lise Buyer, who previously helped guide Google through its IPO and now works as an adviser to companies going public, said there’s no appetite for risk in the market under these conditions. But there is risk that staffers get jittery, and they’ll surely look to their leaders for some reassurance.
“Until markets settle out and we have the opportunity to access valuation levels, public company CEOs should work to calm potentially distressed employees,” Buyer said in an email. “And private company managements should refine plans to get by on dollars already in the treasury.”
— CNBC’s Hayden Field, Jordan Novet, Leslie Picker, Annie Palmer and Samantha Subin contributed to this report.
-
Sports2 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports12 months agoStory injured on diving stop, exits Red Sox game
-
Sports1 year agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike
-
Business3 years agoBank of England’s extraordinary response to government policy is almost unthinkable | Ed Conway