OpenAI and Anthropic agree to let U.S. AI Safety Institute test and evaluate new models
Jakub Porzycki | Nurphoto | Getty Images
OpenAI and Anthropic, the two most richly valued artificial intelligence startups, have agreed to let the U.S. AI Safety Institute test their new models before releasing them to the public, following increased concerns in the industry about safety and ethics in AI.
The institute, housed within the Department of Commerce at the National Institute of Standards and Technology (NIST), said in a press release that it will get “access to major new models from each company prior to and following their public release.”
The group was established after the Biden-Harris administration issued the U.S. government’s first-ever executive order on artificial intelligence in October 2023, requiring new safety assessments, equity and civil rights guidance and research on AI’s impact on the labor market.
“We are happy to have reached an agreement with the US AI Safety Institute for pre-release testing of our future models,” OpenAI CEO Sam Altman wrote in a post on X. OpenAI also confirmed to CNBC on Thursday that, in the past year, the company has doubled its number of weekly active users from late last year to 200 million. Axios was first to report on the number.
The news comes a day after reports surfaced that OpenAI is in talks to raise a funding round valuing the company at more than $100 billion. Thrive Capital is leading the round and will invest $1 billion, according to a source with knowledge of the matter who asked not to be named because the details are confidential.
Anthropic, founded by ex-OpenAI research executives and employees, was most recently valued at $18.4 billion. Anthropic counts Amazon as a leading investor, while OpenAI is heavily backed by Microsoft.
The agreements between the government, OpenAI and Anthropic “will enable collaborative research on how to evaluate capabilities and safety risks, as well as methods to mitigate those risks,” according to Thursday’s release.
Jason Kwon, OpenAI’s chief strategy officer, told CNBC in a statement that, “We strongly support the U.S. AI Safety Institute’s mission and look forward to working together to inform safety best practices and standards for AI models.”
Jack Clark, co-founder of Anthropic, said the company’s “collaboration with the U.S. AI Safety Institute leverages their wide expertise to rigorously test our models before widespread deployment” and “strengthens our ability to identify and mitigate risks, advancing responsible AI development.”
A number of AI developers and researchers have expressed concerns about safety and ethics in the increasingly for-profit AI industry. Current and former OpenAI employees published an open letter on June 4, describing potential problems with the rapid advancements taking place in AI and a lack of oversight and whistleblower protections.
“AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this,” they wrote. AI companies, they added, “currently have only weak obligations to share some of this information with governments, and none with civil society,” and they can not be “relied upon to share it voluntarily.”
Days after the letter was published, a source familiar to the mater confirmed to CNBC that the FTC and the Department of Justice were set to open antitrust investigations into OpenAI, Microsoft and Nvidia. FTC Chair Lina Khan has described her agency’s action as a “market inquiry into the investments and partnerships being formed between AI developers and major cloud service providers.”
On Wednesday, California lawmakers passed a hot-button AI safety bill, sending it to Governor Gavin Newsom’s desk. Newsom, a Democrat, will decide to either veto the legislation or sign it into law by Sept. 30. The bill, which would make safety testing and other safeguards mandatory for AI models of a certain cost or computing power, has been contested by some tech companies for its potential to slow innovation.
WATCH: Google, OpenAI and others oppose California AI safety bill
Technology
Trump aims to cut $6 billion from NASA budget, shifting $1 billion to Mars-focused missions
Nurphoto | Nurphoto | Getty Images
Chinese bargain retailer Temu changed its business model in the U.S. as the Trump administration’s new rules on low-value shipments took effect Friday.
In recent days, Temu has abruptly shifted its website and app to only display listings for products shipped from U.S.-based warehouses. Items shipped directly from China, which previously blanketed the site, are now labeled as out of stock.
Temu made a name for itself in the U.S. as a destination for ultra-discounted items shipped direct from China, such as $5 sneakers and $1.50 garlic presses. It’s been able to keep prices low because of the so-called de minimis rule, which has allowed items worth $800 or less to enter the country duty-free since 2016.
The loophole expired Friday at 12:01 a.m. EDT as a result of an executive order signed by President Donald Trump in April. Trump briefly suspended the de minimis rule in February before reinstating the provision days later as customs officials struggled to process and collect tariffs on a mountain of low-value packages.
The end of de minimis, as well as Trump’s new 145% tariffs on China, has forced Temu to raise prices, suspend its aggressive online advertising push and now alter the selection of goods available to American shoppers to circumvent higher levies.
A Temu spokesperson confirmed to CNBC that all sales in the U.S. are now handled by local sellers and said they are fulfilled “from within the country.” Temu said pricing for U.S. shoppers “remains unchanged.”
“Temu has been actively recruiting U.S. sellers to join the platform,” the spokesperson said. “The move is designed to help local merchants reach more customers and grow their businesses.”
Before the change, shoppers who attempted to purchase Temu products shipped from China were confronted with “import charges” of between 130% and 150%. The fees often cost more than the individual item and more than doubled the price of many orders.
Temu advertises that local products have “no import charges” and “no extra charges upon delivery.”
The company, which is owned by Chinese e-commerce giant PDD Holdings, has gradually built up its inventory in the U.S. over the past year in anticipation of escalating trade tensions and the removal of de minimis.
Shein, which has also benefited from the loophole, moved to raise prices last week. The fast-fashion retailer added a banner at checkout that says, “Tariffs are included in the price you pay. You’ll never have to pay extra at delivery.”
Many third-party sellers on Amazon rely on Chinese manufacturers to source or assemble their products. The company’s Temu competitor, called Amazon Haul, has relied on de minimis to ship products priced at $20 or less directly from China to the U.S.
Amazon said Tuesday following a dustup with the White House that had it considered showing tariff-related costs on Haul products ahead of the de minimis cutoff but that it has since scrapped those plans.
Prior to Trump’s second term in office, the Biden administration had also looked to curtail the provision. Critics of the de minimis provision argue that it harms American businesses and that it facilitates shipments of fentanyl and other illicit substances because, they say, the packages are less likely to be inspected by customs agents.
— CNBC’s Gabrielle Fonrouge contributed to this report.
WATCH: Trump tariffs mean higher prices, big losses for Amazon sellers
Jeff Bezos, founder and executive chairman of Amazon and owner of The Washington Post, takes the stage during The New York Times’ annual DealBook Summit, at Jazz at Lincoln Center in New York City, Dec. 4, 2024.
Michael M. Santiago | Getty Images
Amazon founder Jeff Bezos plans to sell up to 25 million shares in the company over the next year, according to a financial filing on Friday.
Bezos, who stepped down as CEO in 2021 but remains Amazon’s top shareholder, is selling the shares as part of a trading plan adopted on March 4, the filing states. The stake would be worth about $4.8 billion at the current price.
The disclosure follows Amazon’s first-quarter earnings report late Thursday. While profit and revenue topped estimates, the company’s forecast for operating income in the current quarter came in below Wall Street’s expectations.
The results show that Amazon is bracing for uncertainty related to President Donald Trump’s sweeping new tariffs. The company landed in the crosshairs of the White House this week over a report that Amazon planned to show shoppers the cost of the tariffs. Trump personally called Bezos to complain, and Amazon clarified that no such change was coming.
Bezos previously offloaded about $13.5 billion worth of Amazon shares last year, marking his first sale of company stock since 2021.
Since handing over the Amazon CEO role to Andy Jassy, Bezos has spent more of his time on his space exploration company, Blue Origin, and his $10 billion climate and biodiversity fund. He’s used Amazon share sales to help fund Blue Origin, as well as the Day One Fund, which he launched in September 2018 to provide education in low-income communities and combat homelessness.
-
Sports3 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoStory injured on diving stop, exits Red Sox game
-
Sports2 years agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike
-
Business3 years agoBank of England’s extraordinary response to government policy is almost unthinkable | Ed Conway