OpenAI and Anthropic, the two most richly valued artificial intelligence startups, have agreed to let the U.S. AI Safety Institute test their new models before releasing them to the public, following increased concerns in the industry about safety and ethics in AI.
The institute, housed within the Department of Commerce at the National Institute of Standards and Technology (NIST), said in a press release that it will get “access to major new models from each company prior to and following their public release.”
The group was established after the Biden-Harris administration issued the U.S. government’s first-ever executive order on artificial intelligence in October 2023, requiring new safety assessments, equity and civil rights guidance and research on AI’s impact on the labor market.
“We are happy to have reached an agreement with the US AI Safety Institute for pre-release testing of our future models,” OpenAI CEO Sam Altman wrote in a post on X. OpenAI also confirmed to CNBC on Thursday that, in the past year, the company has doubled its number of weekly active users from late last year to 200 million. Axios was first to report on the number.
The news comes a day after reports surfaced that OpenAI is in talks to raise a funding round valuing the company at more than $100 billion. Thrive Capital is leading the round and will invest $1 billion, according to a source with knowledge of the matter who asked not to be named because the details are confidential.
Anthropic, founded by ex-OpenAI research executives and employees, was most recently valued at $18.4 billion. Anthropic counts Amazon as a leading investor, while OpenAI is heavily backed by Microsoft.
The agreements between the government, OpenAI and Anthropic “will enable collaborative research on how to evaluate capabilities and safety risks, as well as methods to mitigate those risks,” according to Thursday’s release.
Jason Kwon, OpenAI’s chief strategy officer, told CNBC in a statement that, “We strongly support the U.S. AI Safety Institute’s mission and look forward to working together to inform safety best practices and standards for AI models.”
Jack Clark, co-founder of Anthropic, said the company’s “collaboration with the U.S. AI Safety Institute leverages their wide expertise to rigorously test our models before widespread deployment” and “strengthens our ability to identify and mitigate risks, advancing responsible AI development.”
A number of AI developers and researchers have expressed concerns about safety and ethics in the increasingly for-profit AI industry. Current and former OpenAI employees published an open letter on June 4, describing potential problems with the rapid advancements taking place in AI and a lack of oversight and whistleblower protections.
“AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this,” they wrote. AI companies, they added, “currently have only weak obligations to share some of this information with governments, and none with civil society,” and they can not be “relied upon to share it voluntarily.”
Days after the letter was published, a source familiar to the mater confirmed to CNBC that the FTC and the Department of Justice were set to open antitrust investigations into OpenAI, Microsoft and Nvidia. FTC Chair Lina Khan has described her agency’s action as a “market inquiry into the investments and partnerships being formed between AI developers and major cloud service providers.”
On Wednesday, California lawmakers passed a hot-button AI safety bill, sending it to Governor Gavin Newsom’s desk. Newsom, a Democrat, will decide to either veto the legislation or sign it into law by Sept. 30. The bill, which would make safety testing and other safeguards mandatory for AI models of a certain cost or computing power, has been contested by some tech companies for its potential to slow innovation.
The replica of the ARM is an electronic chip board during a collaborative ceremony launching a partnership between Malaysia and ARM Holdings in Kuala Lumpur, Malaysia, on March 5, 2025.
Hari Anggara | Nurphoto | Getty Images
Arm Holdings shares dipped as much as 9% in after-hours trading on the company’s first-quarter earnings results Wednesday.
Here’s how the company did, compared with estimates from analysts polled by LSEG:
Earnings per share: 35 cents vs. 35 cents expected.
Revenue: $1.05 billion vs. $1.06 billion expected.
The company said it expects second-quarter revenue in the range of $1.01 billion to $1.11 billion, which was in line with $1.05 billion expected by analysts tracked by LSEG.
ARM is a chip technology firm that sells architecture for making chips that power billions of devices, including Apple and Qualcomm‘s chips.
During the quarter, Samsung launched the Galaxy Flip 7 based on the Exynos 2500, built on Arm’s compute subsystem platform.
CEO Rene Haas said in an interview with Reuters that the company was “consciously deciding to invest more heavily,” suggesting the company is considering designing its own processors.
Cristiano Amon, CEO & President, Qualcomm, on Centre Stage during day one of Web Summit 2024 at the MEO Arena in Lisbon, Portugal.
Shauna Clinton | Sportsfile | Getty Images
Qualcomm reported fiscal third-quarter earnings on Wednesday that beat Wall Street expectations and provided a stronger-than-expected guide for the current quarter. Qualcomm shares slid in extended trading.
Here’s how the chipmaker did for the quarter ending June 29 compared to LSEG consensus expectations:
Earnings per share: $2.77 adjusted versus $2.71 expected
Revenue: $10.37 billion versus $10.35 billion expected
In the current quarter, Qualcomm said it expected $2.85 per share at the midpoint of adjusted earnings on $10.7 billion in revenue at the midpoint. Analysts polled by LSEG were expecting $2.83 in adjusted earnings per share on $10.35 billion in revenue.
Net income during the quarter ending in June was $2.66 billion, or $2.43 per share, versus $2.13 billion, or $1.88 per share a year ago.
Qualcomm’s most important business is selling chips for smartphones under its Snapdragon brand, including the central processor and modem for high-end devices made by Samsung. It also provides modems to Apple. Its handset chip business reported $6.33 billion in revenue during the quarter, just shy of Wall Street expectations of $6.44 billion.
Qualcomm expects to lose Apple as a customer for its modem business in the coming years. But the company has been working to diversify its business by making chips for other devices, including Windows PCs and Meta‘s Quest virtual-reality headsets and Meta Ray-Bans smart glasses.
Read more CNBC tech news
Qualcomm CEO Cristiano Amon highlighted the company’s work with Meta in a short interview on Wednesday.
He said that making chips for devices like Meta’s Ray-Bans smart glasses was a good example of the chipmaker’s AI strategy, which was to embrace “personal AI,” or AI applications that run on devices, not the cloud.
Qualcomm reports its Meta revenues under its “Internet of Things” division, which had $1.68 billion in revenue during the quarter.
Amon referenced Mark Zuckerberg‘s AI vision statement Wednesday that focused on “personal superintelligence,” saying “the upside we had in the quarter within IoT is what we do in with smart glasses.”
CFO Akash Palkhiwala said that Meta had stronger-than-expected chip consumption during the quarter.
On Monday, Ray-Ban parent EssilorLuxottica said that sales of the smart glasses more than tripled on an annual basis.
“Mark put out a video today, just with a very clear vision of how they see personal AI and super intelligence evolving, and we are a key part of making that division happen,” Palkhiwala said.
Ray-Ban Meta smart glasses are powered by a Qualcomm chip. Qualcomm, Samsung and Google are working on smart glasses, according to Qualcomm CEO Cristiano Amon.
Nurphoto | Nurphoto | Getty Images
Amon also said Qualcomm would start to provide data about how much its chip business is growing without Apple — about 15% this year, he said.
Qualcomm is also looking to expand into data centers and sell versions of its chips that can be used for deploying artificial intelligence, Amon said on a call with an analysts. He said that Qualcomm was already in discussions with a major cloud company — called a hyperscaler — to supply AI chips. He said that Qualcomm could start to see revenues in its fiscal 2028.
“While we are in the early stages of this expansion, we are engaged with multiple potential customers,” Among said. “We are currently in advanced discussions with a leading hyperscaler.”
The company’s automotive business has been highlighted by Amon as one of the biggest growth opportunities for the company, but in the third quarter, it grew 21% to $984 million, below the 24% growth rate of the company’s IoT business.
Qualcomm’s other major division is QTL, which includes licensing fees for technology that Qualcomm developed and patented, including parts of the 5G standard. Overall, QTL revenues rose 11% to $1.32 billion.
Qualcomm said it spent just under $1 billion on cash dividends and $2.8 billion repurchasing 19 million shares of its stock during the quarter.
Meta CEO Mark Zuckerberg presents Orion AR Glasses as he makes a keynote speech during the Meta Connect annual event at the company’s headquarters in Menlo Park, California, on Sept. 25, 2024.
Manuel Orbegozo | Reuters
Meta’s Reality Labs, the unit tasked with building the futuristic metaverse, continues bleeding money.
The social media company reported its second-quarter earnings on Wednesday and revealed that Reality Labs logged an operating loss of $4.53 billion while recording $370 million in sales during the period. Analysts were projecting that unit to post a second-quarter operating loss of $4.99 billion while generating $381 million in sales.
The Reality Labs division oversees the Quest line of virtual reality headsets in addition to the Ray-Ban Meta smart glasses, which are jointly developed with the French-Italian eyewear giant EssilorLuxottica. Meta wants Reality Labs to create cutting-edge products similar to the prototype Orion augmented reality glasses that could underpin a new, immersive computing platform.
But developing VR, AR and other new devices is an expensive endeavor, with the Reality Labs division logging nearly $70 billion in cumulative losses since late 2020. Meta in April said Reality Labs recorded an operating loss of $4.2 billion during the first quarter while bringing in $412 million in sales.
Although the Quest VR headsets haven’t become breakout hits, the Ray-Ban Meta smart glasses are showing signs of success.
EssilorLuxottica on Monday said Ray-Ban Meta smart glasses sales more than tripled year over year for the first half of 2025. The eyewear giant and Meta debuted in June the new Oakley Meta smart glasses, which is the latest product spawned from their partnership.
Meta said in April that an undisclosed number of Reality Labs employees who were part of its Oculus Studios VR and AR software unit were laid off.