OpenAI and Anthropic agree to let U.S. AI Safety Institute test and evaluate new models

Published

11 months ago

August 29, 2024

admin

Jakub Porzycki | Nurphoto | Getty Images

OpenAI and Anthropic, the two most richly valued artificial intelligence startups, have agreed to let the U.S. AI Safety Institute test their new models before releasing them to the public, following increased concerns in the industry about safety and ethics in AI.

The institute, housed within the Department of Commerce at the National Institute of Standards and Technology (NIST), said in a press release that it will get “access to major new models from each company prior to and following their public release.”

The group was established after the Biden-Harris administration issued the U.S. government’s first-ever executive order on artificial intelligence in October 2023, requiring new safety assessments, equity and civil rights guidance and research on AI’s impact on the labor market.

“We are happy to have reached an agreement with the US AI Safety Institute for pre-release testing of our future models,” OpenAI CEO Sam Altman wrote in a post on X. OpenAI also confirmed to CNBC on Thursday that, in the past year, the company has doubled its number of weekly active users from late last year to 200 million. Axios was first to report on the number.

The news comes a day after reports surfaced that OpenAI is in talks to raise a funding round valuing the company at more than $100 billion. Thrive Capital is leading the round and will invest $1 billion, according to a source with knowledge of the matter who asked not to be named because the details are confidential.

Anthropic, founded by ex-OpenAI research executives and employees, was most recently valued at $18.4 billion. Anthropic counts Amazon as a leading investor, while OpenAI is heavily backed by Microsoft.

The agreements between the government, OpenAI and Anthropic “will enable collaborative research on how to evaluate capabilities and safety risks, as well as methods to mitigate those risks,” according to Thursday’s release.

Jason Kwon, OpenAI’s chief strategy officer, told CNBC in a statement that, “We strongly support the U.S. AI Safety Institute’s mission and look forward to working together to inform safety best practices and standards for AI models.”

Jack Clark, co-founder of Anthropic, said the company’s “collaboration with the U.S. AI Safety Institute leverages their wide expertise to rigorously test our models before widespread deployment” and “strengthens our ability to identify and mitigate risks, advancing responsible AI development.”

A number of AI developers and researchers have expressed concerns about safety and ethics in the increasingly for-profit AI industry. Current and former OpenAI employees published an open letter on June 4, describing potential problems with the rapid advancements taking place in AI and a lack of oversight and whistleblower protections.

“AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this,” they wrote. AI companies, they added, “currently have only weak obligations to share some of this information with governments, and none with civil society,” and they can not be “relied upon to share it voluntarily.”

Days after the letter was published, a source familiar to the mater confirmed to CNBC that the FTC and the Department of Justice were set to open antitrust investigations into OpenAI, Microsoft and Nvidia. FTC Chair Lina Khan has described her agency’s action as a “market inquiry into the investments and partnerships being formed between AI developers and major cloud service providers.”

On Wednesday, California lawmakers passed a hot-button AI safety bill, sending it to Governor Gavin Newsom’s desk. Newsom, a Democrat, will decide to either veto the legislation or sign it into law by Sept. 30. The bill, which would make safety testing and other safeguards mandatory for AI models of a certain cost or computing power, has been contested by some tech companies for its potential to slow innovation.

WATCH: Google, OpenAI and others oppose California AI safety bill

Google, OpenAI & others oppose California AI safety bill

Technology

Spotify stock falls on revenue miss, lackluster guidance

Published

5 hours ago

July 29, 2025

admin

Spotify stock falls on revenue miss, lackluster guidance

Thomas Fuller | Lightrocket | Getty Images

Spotify shares dropped about 4% Tuesday after the music streaming platform fell short of Wall Street’s expectations and posted weak guidance for the current quarter.

Here’s how the company did versus LSEG estimates:

Loss: Loss of .42 euros vs earnings of 1.90 euros per share expected
Revenue: 4.19 billion euros vs. 4.26 billion expected

The Sweden-based music platform’s revenues rose 10% from about 3.81 billion euros in the year-ago period. The company posted a net loss of 86 million euros, or a loss of .42 euros per share, down from net income of 225 million euros, or 1.10 euros per share a year ago.

Third-quarter guidance came up short of Wall Street’s forecast.

The company expects revenues to reach 4.2 billion euros, compared to a 4.47 billion euro estimate from StreetAccount. Spotify said the forecast accounts for a 490-basis-point headwind due to foreign exchange rates.

Samsung backs South Korean AI chip startup Rebellions ahead of IPO

Published

8 hours ago

July 29, 2025

admin

The Rebel-Quad is the second-generation product from Rebellions and is made up of four Rebel AI chips. Rebellions, a South Korean firm, is looking to rival companies like Nvidia in AI chips.

Rebellions

South Korean artificial intelligence chip startup Rebellions has raised money from tech giant Samsung and is targeting a funding round of up to $200 million ahead of a public listing, the company’s management told CNBC on Tuesday.

Last year, Rebellions merged with another startup in South Korea called Sapeon, creating a firm that is being positioned as one of the country’s promising rivals to Nvidia.

Rebellions is currently raising money and is targeting funding of between $150 million and $200 million, Sungkyue Shin, chief financial officer of the startup, told CNBC on Tuesday.

Samsung’s investment in Rebellions last week was part of that, Shin said, though he declined to say how much the tech giant poured in.

Since its founding in 2020, Rebellions has raised $220 million, Shin added.

The current funding round is ongoing and Shin said Rebellions is talking to its current investors as well as investors in Korea and globally to participate in the capital raise. Rebellions has some big investors, including South Korean chip giant SK Hynix, telecommunication firms SK Telecom and Korea Telecom, and Saudi Arabian oil giant Aramco.

AI chip startup Rebellions looks to raise up to $200 million ahead of IPO

Rebellions was last valued at $1 billion. Shin said the current round of funding would push the valuation over $1 billion but declined to give specific figure.

Rebellions is aiming for an initial public offering once this funding round has closed.

“Our master plan is going public,” Shin said.

Rebellions designs chips that are focused on AI inferencing rather than training. Inferencing is when a pre-trained AI model interprets live data to come up with a result, much like the answers that are produced by popular chatbots.

With the backing of major South Korean firms and investors, Rebellions is hoping to make a global play where it will look to challenge Nvidia and AMD as well as a slew of other startups in the inferencing space.

Samsung collaboration

Rebellions has been working with Samsung to bring its second-generation chip, Rebel, to market. Samsung owns a chip manufacturing business, also known as foundry. Four Rebel chips are put together to make the Rebel-Quad, the product that Rebellions will eventually sell. A Rebellions spokesperson said the chip will be launched later this year.

The funding will partly go toward Rebellions’ product development. Rebellions is currently testing its chip which will eventually be produced on a larger scale by Samsung.

“Initial results have been very promising,” Sunghyun Park, CEO of Rebellions, told CNBC on Tuesday.

South Korean AI startup Rebellions says tariffs could delay IPO by 'a little bit'

Park said Samsung invested in Rebellions partly because of the the good results that the chip has so far produced.

Samsung is manufacturing Rebellions’ semiconductor using its 4 nanometer process, which is among the leading-edge chipmaking nodes. For comparison, Nvidia’s current Blackwell chips use the 4 nanometer process from Taiwan Semiconductor Manufacturing Co. Rebellions will also use Samsung’s high bandwidth memory, known as HBM3e. This type of memory is stacked and is required to handle large data processing loads.

That could turn out to be a strategic win for Samsung, which is a very distant second to TSMC in terms of market share in the foundry business. Samsung has been looking to boost its chipmaking division. Samsung Electronics recently entered into a $16.5 billion contract for supplying semiconductors to Tesla.

If Rebellions manages to find a large customer base, this could give Samsung a major customer for its foundry business.

Technology

Europe sets its sights on multi-billion-euro gigawatt factories as it plays catch-up on AI

Published

11 hours ago

July 29, 2025

admin

Europe sets its sights on multi-billion-euro gigawatt factories as it plays catch-up on AI

Data storage tapes are stored at the National Energy Research Scientific Computing Center (NERSC) facility at the Lawrence Berkeley National Laboratory, which will house the U.S. supercomputer to be powered by Nvidia’s forthcoming Vera Rubin chips, in Berkeley, California, U.S. May 29, 2025.

Manuel Orbegozo | Reuters

Europe is setting its sights on gigawatt factories in a bid to bolster its lagging artificial intelligence industry and meet the challenges of a rapidly-changing sector.

Buzz around the concept of factories that industrialize manufacturing AI has gained ground in recent months, particularly as Nvidia CEO Jensen Huang stressed the importance of the infrastructure at a June event. Huang hailed a new “industrial revolution” at the GTC conference in Paris, France, and said his firm was working to help countries build revenue-generating AI factories through partnerships in France, Italy and the U.K.

For its part, the European Union describes the factories as a “dynamic ecosystem” that brings together computing power, data and talent to create AI models and applications.

The bloc has long been a laggard behind the U.S. and China in the race to scale up artificial intelligence. With 27 members in the union, the region is slower to act when it comes to agreeing new legislation. Higher energy costs, permitting delays and a grid in dire need of modernization can also hamper developments.

Henna Virkkunen, the European Commission’s executive vice president for tech sovereignty, told CNBC that the bloc’s goal is to bring together high quality data sets, computing capacity and researchers, all in one place.

“We have, for example, 30% more researchers per capita than the U.S. has, focused on AI. Also we have around 7,000 startups [that] are developing AI, but the main obstacle for them is that they have very limited computing capacity. And that’s why we decided that, together with our member states, we are investing in this very crucial infrastructure,” she said.

These are very big investments because they are four times more powerful when it comes to computing capacities than the biggest AI factories.

Henna Virkkunen

European Commission’s executive vice president for tech sovereignty

“We have everything what is needed to be competitive in this sector, but at the same time we want to build up our technological sovereignty and our competitiveness.”

So far, the EU has put up 10 billion euros ($11.8 billion) in funding to set up 13 AI factories and 20 billion euros as a starting point for investment in the gigafactories, marking what it says is the “largest public investment in AI in the world.” The bloc has already received 76 expressions of interest in the gigafactories from 16 member states across 60 sites, Virkkunen said.

The call for interest in gigafactories was “overwhelming,” going far beyond the bloc’s expectations, Virkkunen noted. However, in order for the factories to make a noteworthy addition to Europe’s computing capacity, significantly more investment will be required from the private sector to fund the expensive infrastructure.

‘Intelligence revolution’

The EU describes the facilities as a “one-stop shop” for AI firms. They’re intended to mirror the process carried out in industrial factories, which transform raw materials into goods and services. With an AI factory, raw data goes into the input, and advanced AI products are the expected outcome.

It’s essentially a data center with additional infrastructure related to how the technology will be adopted, according to Andre Kukhnin, equity research analyst at UBS.

“The idea is to create GPU [graphics processing units] capacity, so to basically build data centers with GPUs that can train models and run inference… and then to create an infrastructure that allows you to make this accessible to SMEs and parties that would not be able to just go and build their own,” Kukhnin said.

How the facility will be used is key to its designation as an AI factory, adds Martin Wilkie, research analyst at Citi.

“You’re creating a platform by having these chips that have insane levels of compute capacity,” he said. “And if you’ve attached it to a grid that is able to get the power to actually use them to full capacity, then the world is at your feet. You have this enormous ability to do something, but what the success of it is, will be defined by what you use it for.”

Telecommunications firm Telenor is already exploring possible use cases for such facilities with the launch of its AI factory in Norway in November last year. The company currently has a small cluster of GPUs up and running, as it looks to test the market before scaling up.

Telenor’s Chief Innovation Officer and Head of the AI Factory Kaaren Hilsen and EVP Infrastructure Jannicke Hilland in front of a Nvidia rack at the firm’s AI factory

Telenor

“The journey started with a belief — Nvidia had a belief that every country needs to produce its own intelligence,” Telenor’s Chief Innovation Officer and Head of the AI Factory Kaaren Hilsen told CNBC.

Hilsen stressed that data sovereignty is key. “If you want to use AI to innovate and to make business more efficient, then you’re potentially putting business critical and business sensitive information into these AI models,” she said.

The company is working with BabelSpeak, which Hilsen described as a Norwegian version of ChatGPT. The technology translates sensitive dialogues, such as its pilot with the border police who can’t use public translation services because of security issues.

We’re experiencing an “intelligence revolution” whereby “sovereign AI factories can really help advance society,” Hilsen said.

Billion-euro investments

Virkkunen said the region’s first AI factory will be operational in coming weeks, with one of the biggest projects launching in Munich, Germany in the first days of September. It’s a different story for the gigafactories.

“These are very big investments because they are four times more powerful when it comes to computing capacities than the biggest AI factories, and it means billions in investments. Each of these need three to five billion [euros] in investment,” the commissioner said, adding that the bloc will look to set up a consortium of partners and then officially open a call for investment later this year.

Bertin Martens, senior research fellow at Bruegel, questioned why such investments needed to subsidized by government funds.

“We don’t know yet how much private investment has been proposed as a complement to the taxpayer subsidy, and what capacity and how big these factories are. This is still very much unclear at this stage, so it’s very hard to say how much this will add in terms of computing capacity,” he said.

Power consumption is also a key issue. Martens noted that building an AI gigafactory may take one to two years — but building a power generation of that size requires much more time.

“If you want to build a state-of-the-art gigafactory with hundreds of thousands of Nvidia chips, you have to count on the power consumption of at least one gigawatt for one of those factories. Whether there’s enough space in Europe’s electricity grid in all of these countries to create those factories remains to be seen… this will require major investment in power regeneration capacity,” he told CNBC.

UBS forecasts that the current installed global data center capacity of 85 GW will double due to soaring demand. Based on the EU’s 20-billion-euro investment and the plan for each factory to run 100,000 advanced processors, UBS estimates each factory could be around 100-150 MW with a total capacity for all of the facilities of around 1.5-2 GW.

That could add around 15% to Europe’s total capacity — a sizeable boost, even when compared to the U.S., which currently owns around a third of global capacity, according to the data.

Following the announcement of the EU-U.S. trade framework, EU chief Ursula von der Leyen said Sunday that U.S. AI chips will help power the bloc’s AI gigafactories in a bid to help the States “maintain their technological edge.”

“One could argue that it’s relatively easy, provided you have the money. It’s relatively easy to buy the chips from Nvidia and to create these hardware factories, but to make it run and to make it economically viable is a completely different question,” Martens told CNBC.

He said that the EU will likely have to start at a smaller scale, as the region is unable to immediately build its own frontier models in AI because of their expense.

“I think in time, Europe can gradually build up its infrastructure and its business models around AI to reach that stage, but that will not happen immediately,” Martens said.