OpenAI and Anthropic, the two most richly valued artificial intelligence startups, have agreed to let the U.S. AI Safety Institute test their new models before releasing them to the public, following increased concerns in the industry about safety and ethics in AI.
The institute, housed within the Department of Commerce at the National Institute of Standards and Technology (NIST), said in a press release that it will get “access to major new models from each company prior to and following their public release.”
The group was established after the Biden-Harris administration issued the U.S. government’s first-ever executive order on artificial intelligence in October 2023, requiring new safety assessments, equity and civil rights guidance and research on AI’s impact on the labor market.
“We are happy to have reached an agreement with the US AI Safety Institute for pre-release testing of our future models,” OpenAI CEO Sam Altman wrote in a post on X. OpenAI also confirmed to CNBC on Thursday that, in the past year, the company has doubled its number of weekly active users from late last year to 200 million. Axios was first to report on the number.
The news comes a day after reports surfaced that OpenAI is in talks to raise a funding round valuing the company at more than $100 billion. Thrive Capital is leading the round and will invest $1 billion, according to a source with knowledge of the matter who asked not to be named because the details are confidential.
Anthropic, founded by ex-OpenAI research executives and employees, was most recently valued at $18.4 billion. Anthropic counts Amazon as a leading investor, while OpenAI is heavily backed by Microsoft.
The agreements between the government, OpenAI and Anthropic “will enable collaborative research on how to evaluate capabilities and safety risks, as well as methods to mitigate those risks,” according to Thursday’s release.
Jason Kwon, OpenAI’s chief strategy officer, told CNBC in a statement that, “We strongly support the U.S. AI Safety Institute’s mission and look forward to working together to inform safety best practices and standards for AI models.”
Jack Clark, co-founder of Anthropic, said the company’s “collaboration with the U.S. AI Safety Institute leverages their wide expertise to rigorously test our models before widespread deployment” and “strengthens our ability to identify and mitigate risks, advancing responsible AI development.”
A number of AI developers and researchers have expressed concerns about safety and ethics in the increasingly for-profit AI industry. Current and former OpenAI employees published an open letter on June 4, describing potential problems with the rapid advancements taking place in AI and a lack of oversight and whistleblower protections.
“AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this,” they wrote. AI companies, they added, “currently have only weak obligations to share some of this information with governments, and none with civil society,” and they can not be “relied upon to share it voluntarily.”
Days after the letter was published, a source familiar to the mater confirmed to CNBC that the FTC and the Department of Justice were set to open antitrust investigations into OpenAI, Microsoft and Nvidia. FTC Chair Lina Khan has described her agency’s action as a “market inquiry into the investments and partnerships being formed between AI developers and major cloud service providers.”
On Wednesday, California lawmakers passed a hot-button AI safety bill, sending it to Governor Gavin Newsom’s desk. Newsom, a Democrat, will decide to either veto the legislation or sign it into law by Sept. 30. The bill, which would make safety testing and other safeguards mandatory for AI models of a certain cost or computing power, has been contested by some tech companies for its potential to slow innovation.
Amazon logo on a brick building exterior, San Francisco, California, August 20, 2024.
Smith Collection | Gado | Archive Photos | Getty Images
Amazon representatives met with the House China committee in recent months to discuss lawmaker concerns over the company’s partnership with TikTok, CNBC confirmed.
A spokesperson for the House Select Committee on the Chinese Communist Party confirmed the meeting, which centered on a shopping deal between Amazon and TikTok announced in August. The agreement allows users of TikTok, owned by China’s ByteDance, to link their account with Amazon and make purchases from the site without leaving TikTok.
“The Select Committee conveyed to Amazon that it is dangerous and unwise for Amazon to partner with TikTok given the grave national security threat the app poses,” the spokesperson said. The parties met in September, according to Bloomberg, which first reported the news.
Representatives from Amazon and TikTok did not immediately respond to CNBC’s request for comment.
TikTok’s future viability in the U.S. is uncertain. In April, President Joe Biden signed a law that requires ByteDance to sell TikTok by Jan. 19. If TikTok fails to cut ties with its parent company, app stores and internet hosting services would be prohibited from offering the app.
President-elect Donald Trump could rescue TikTok from a potential U.S. ban. He promised on the campaign trail that he would “save” TikTok, and said in a March interview with CNBC’s “Squawk Box” that “there’s a lot of good and there’s a lot of bad” with the app.
In his first administration, Trump had tried to implement a TikTok ban. He changed his stance around the time he met with billionaire Jeff Yass. The Republican megadonor’s trading firm, Susquehanna International Group, owns a 15% stake in ByteDance, while Yass has a 7% stake in the company, NBC and CNBC reported in March.
— CNBC’s Jonathan Vanian contributed to this report.
A worker delivers Amazon packages in San Francisco on Oct. 24, 2024.
David Paul Morris | Bloomberg | Getty Images
Amazon on Thursday announced Prime members can access new fixed pricing for treatment of conditions like erectile dysfunction and men’s hair loss, its latest effort to compete with other direct-to-consumer marketplaces such as Hims & Hers Health and Ro.
Shares of Hims & Hers fell as much as 17% on Thursday, on pace for its worst day.
Amazon said in a blog post that Prime members can see the cost of a telehealth visit and their desired treatment before they decide to proceed with care for five common issues. Patients can access treatment for anti-aging skin care starting at $10 a month; motion sickness for $2 per use; erectile dysfunction at $19 a month; eyelash growth at $43 a month, and men’s hair loss for $16 a month by using Amazon’s savings benefit Prime Rx at checkout.
Amazon acquired primary care provider One Medical for roughly $3.9 billion in July 2022, and Thursday’s announcement builds on its existing pay-per-visit telehealth offering. Video visits through the service cost $49, and messaging visits cost $29 where available. Users can get treatment for more than 30 common conditions, including sinus infection and pink eye.
Medications filled through Amazon Pharmacy are eligible for discounted pricing and will be delivered to patients’ doors in standard Amazon packaging. Prime members will pay for the consultation and medication, but there are no additional fees, the blog post said.
Amazon has been trying to break into the lucrative health-care sector for years. The company launched its own online pharmacy in 2020 following its acquisition of PillPack in 2018. Amazon introduced, and later shuttered, a telehealth service called Amazon Care, as well as a line of health and wellness devices.
The company has also discontinued a secretive effort to develop an at-home fertility tracker, CNBC reported Wednesday.
Former U.S. Army intelligence analyst Chelsea Manning says censorship is still “a dominant threat,” advocating for a more decentralized internet to help better protect individuals online.
Her comments come amid ongoing tension linked to online safety rules, with some tech executives recently seeking to push back over content moderation concerns.
Speaking to CNBC’s Karen Tso at the Web Summit tech conference in Lisbon, Portugal, on Wednesday, Manning said that one way to ensure online privacy could be “decentralized identification,” which gives individuals the ability to control their own data.
“Censorship is a dominant threat. I think that it is a question of who’s doing the censoring, and what the purpose is — and also censorship in the 21st century is more about whether or not you’re boosted through like an algorithm, and how the fine-tuning of that seems to work,” Manning said.
“I think that social media and the monopolies of social media have sort of gotten us used to the fact that certain things that drive engagement will be attractive,” she added.
“One of the ways that we can sort of countervail that is to go back to the more decentralized and distribute the internet of the early ’90s, but make that available to more people.”
Nym Technologies Chief Security Officer Chelsea Manning at a press conference held with Nym Technologies CEO Harry Halpin in the Media Village to present NymVPN during the second day of Web Summit on November 13, 2024 in Lisbon, Portugal.
Asked how tech companies could make money in such a scenario, Manning said there would have to be “a better social contract” put in place to determine how information is shared and accessed.
“One of the things about distributed or decentralized identification is that through encryption you’re able to sort of check the box yourself, instead of having to depend on the company to provide you with a check box or an accept here, you’re making that decision from a technical perspective,” Manning said.
‘No longer secrecy versus transparency’
Manning, who works as a security consultant at Nym Technologies, a company that specializes in online privacy and security, was convicted of espionage and other charges at a court-martial in 2013 for leaking a trove of secret military files to online media publisher WikiLeaks.
She was sentenced to 35 years in prison, but was later released in 2017, when former U.S. President Barack Obama commuted her sentence.
Asked to what extent the environment has changed for whistleblowers today, Manning said, “We’re at an interesting time because information is everywhere. We have more information than ever.”
She added, “Countries and governments no longer seem to invest the same amount of time and effort in hiding information and keeping secrets. What countries seem to be doing now is they seem to be spending more time and energy spreading misinformation and disinformation.”
Manning said the challenge for whistleblowers now is to sort through the information to understand what is verifiable and authentic.
“It’s no longer secrecy versus transparency,” she added.