OpenAI and Anthropic agree to let U.S. AI Safety Institute test and evaluate new models
Jakub Porzycki | Nurphoto | Getty Images
OpenAI and Anthropic, the two most richly valued artificial intelligence startups, have agreed to let the U.S. AI Safety Institute test their new models before releasing them to the public, following increased concerns in the industry about safety and ethics in AI.
The institute, housed within the Department of Commerce at the National Institute of Standards and Technology (NIST), said in a press release that it will get “access to major new models from each company prior to and following their public release.”
The group was established after the Biden-Harris administration issued the U.S. government’s first-ever executive order on artificial intelligence in October 2023, requiring new safety assessments, equity and civil rights guidance and research on AI’s impact on the labor market.
“We are happy to have reached an agreement with the US AI Safety Institute for pre-release testing of our future models,” OpenAI CEO Sam Altman wrote in a post on X. OpenAI also confirmed to CNBC on Thursday that, in the past year, the company has doubled its number of weekly active users from late last year to 200 million. Axios was first to report on the number.
The news comes a day after reports surfaced that OpenAI is in talks to raise a funding round valuing the company at more than $100 billion. Thrive Capital is leading the round and will invest $1 billion, according to a source with knowledge of the matter who asked not to be named because the details are confidential.
Anthropic, founded by ex-OpenAI research executives and employees, was most recently valued at $18.4 billion. Anthropic counts Amazon as a leading investor, while OpenAI is heavily backed by Microsoft.
The agreements between the government, OpenAI and Anthropic “will enable collaborative research on how to evaluate capabilities and safety risks, as well as methods to mitigate those risks,” according to Thursday’s release.
Jason Kwon, OpenAI’s chief strategy officer, told CNBC in a statement that, “We strongly support the U.S. AI Safety Institute’s mission and look forward to working together to inform safety best practices and standards for AI models.”
Jack Clark, co-founder of Anthropic, said the company’s “collaboration with the U.S. AI Safety Institute leverages their wide expertise to rigorously test our models before widespread deployment” and “strengthens our ability to identify and mitigate risks, advancing responsible AI development.”
A number of AI developers and researchers have expressed concerns about safety and ethics in the increasingly for-profit AI industry. Current and former OpenAI employees published an open letter on June 4, describing potential problems with the rapid advancements taking place in AI and a lack of oversight and whistleblower protections.
“AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this,” they wrote. AI companies, they added, “currently have only weak obligations to share some of this information with governments, and none with civil society,” and they can not be “relied upon to share it voluntarily.”
Days after the letter was published, a source familiar to the mater confirmed to CNBC that the FTC and the Department of Justice were set to open antitrust investigations into OpenAI, Microsoft and Nvidia. FTC Chair Lina Khan has described her agency’s action as a “market inquiry into the investments and partnerships being formed between AI developers and major cloud service providers.”
On Wednesday, California lawmakers passed a hot-button AI safety bill, sending it to Governor Gavin Newsom’s desk. Newsom, a Democrat, will decide to either veto the legislation or sign it into law by Sept. 30. The bill, which would make safety testing and other safeguards mandatory for AI models of a certain cost or computing power, has been contested by some tech companies for its potential to slow innovation.
WATCH: Google, OpenAI and others oppose California AI safety bill
A Waymo rider-only robotaxi is seen during a test ride in San Francisco, California, U.S., December 9, 2022.
Paresh Dave | Reuters
Alphabet’s Waymo unit plans on bringing its robotaxi service to Dallas next year, adding to a growing list of prospective U.S. markets for 2026, including Miami and Washington, D.C.
Rental car company Avis Budget Group will be managing the Waymo fleet in Dallas, via a new partnership the companies announced Monday.
Avis CEO Brian Choi said in a statement that the agreement marks a “milestone” for the company, which is now also working to become “a leading provider of fleet management, infrastructure and operations to the broader mobility ecosystem.”
Waymo robotaxi testing is already underway in downtown Dallas involving the company’s Jaguar I-PACE electric vehicles with the Waymo Driver system. That combines automated driving software, sensors and other hardware that power the vehicles’ “level 4,” driverless operations.
Passengers will be able to hail a driverless ride using the Waymo app in Dallas. In some other markets, Waymo only makes its services available through ride-hailing platform Uber.
Waymo has surged ahead in the robotaxi market while other autonomous vehicle developers, including Tesla, Amazon-owned Zoox, and venture-backed startups such as Nuro, May Mobility and Wayve, are working to make autonomous transportation a commercial reality in the U.S.
Waymo says it conducts more than 250,000 paid weekly trips in the markets where it operates commercially, including Atlanta, Austin, Los Angeles, Phoenix and San Francisco.
Waymo’s steepest competition internationally comes from Baidu’s robotaxi venture Apollo Go in China, which is eyeing expansion in Europe.
On Alphabet’s second-quarter earnings call, execs boasted that, “The Waymo Driver has now autonomously driven over 100 million miles on public roads, and the team is testing across more than 10 cities this year, including New York and Philadelphia.”
The business has become significant enough that Alphabet even added a category to its Other Bets revenue description in its latest quarterly filing.
“Revenues from Other Bets are generated primarily from the sale of autonomous transportation services, healthcare-related services and internet services,” the filing said.
The Other Bets segment remains relatively small, however, with revenue coming in at $373 million in the quarter, up from $365 million a year ago. The division still reported a loss of $1.25 billion, widening from $1.13 billion in the second quarter of 2024.
WATCH: Waymo co-CEO on 10 million driverless rides and Tesla’s coming robtaxi challenge
Ray-Ban Meta smart glasses on display in the window of a Ray Ban store in London, UK, on Friday, July 19, 2024.
Bloomberg | Bloomberg | Getty Images
Revenue from sales of Ray-Ban Meta smart glasses more than tripled year over year, EssilorLuxottica revealed Monday as part of the company’s most recent earnings report.
EssilorLuxottica said the success of the Ray-Ban Meta glasses, built via a partnership with the Facebook parent stemming back to 2019, contributed to its first-half overall sales of 14.02 billion euro (US$16.25 billion), which represents a 7.3% year-over-year jump.
“We are leading the transformation of glasses as the next computing platform, one where AI, sensory tech and a data-rich healthcare infrastructure will converge to empower humans and unlock our full potential,” EssilorLuxottica CEO Francesco Milleri and deputy CEO Paul du Saillant said in a joint-statement. “The success of Ray-Ban Meta, the launch of Oakley Meta Performance AI glasses and the positive response to Nuance Audio are major milestones for us in this new frontier.”
In the earnings report, the company said that its new Oakley Meta smart glasses, unveiled in June, represents the latest product line to come from its partnership with the social media company. CNBC reported in June that Meta and Luxottica plan to debut a Prada-branded version of its smart glasses in the future.
Luxottica owns several well-known brands including Ray-Ban, Oakley, Vogue Eyewear and Persol.
In September, Meta renewed a long-term partnership agreement with Luxottica to “collaborate into the next decade to develop multi-generational smart eyewear products,” according to the announcement.
WATCH: Meta’s ambitious AI plans.
-
Sports3 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoStory injured on diving stop, exits Red Sox game
-
Sports2 years agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Sports2 years agoButton battles heat exhaustion in NASCAR debut
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike