Connect with us

Published

on

OpenAI is disbanding its “AGI Readiness” team, which advised the company on OpenAI’s own capacity to handle increasingly powerful AI and the world’s readiness to manage that technology, according to the head of the team.

On Wednesday, Miles Brundage, senior advisor for AGI Readiness, announced his departure from the company via a Substack post. He wrote that his primary reasons were that the opportunity cost had become too high and he thought his research would be more impactful externally, that he wanted to be less biased and that he had accomplished what he set out to at OpenAI.

Brundage also wrote that, as far as how OpenAI and the world is doing on AGI readiness, “Neither OpenAI nor any other frontier lab is ready, and the world is also not ready.” Brundage plans to start his own nonprofit, or join an existing one, to focus on AI policy research and advocacy. He added that “AI is unlikely to be as safe and beneficial as possible without a concerted effort to make it so.”

Former AGI Readiness team members will be reassigned to other teams, according to the post.

“We fully support Miles’ decision to pursue his policy research outside industry and are deeply grateful for his contributions,” an OpenAI spokesperson told CNBC. “His plan to go all-in on independent research on AI policy gives him the opportunity to have an impact on a wider scale, and we are excited to learn from his work and follow its impact. We’re confident that in his new role, Miles will continue to raise the bar for the quality of policymaking in industry and government.”

In May, OpenAI decided to disband its Superalignment team, which focused on the long-term risks of AI, just one year after it announced the group, a person familiar with the situation confirmed to CNBC at the time.

News of the AGI Readiness team’s disbandment follows the OpenAI board’s potential plans to restructure the firm to a for-profit business, and after three executives — CTO Mira Murati, research chief Bob McGrew and research VP Barret Zoph — announced their departure on the same day last month.

Earlier in October, OpenAI closed its buzzy funding round at a valuation of $157 billion, including the $6.6 billion the company raised from an extensive roster of investment firms and big tech companies. It also received a $4 billion revolving line of credit, bringing its total liquidity to more than $10 billion. The company expects about $5 billion in losses on $3.7 billion in revenue this year, CNBC confirmed with a source familiar last month.

And in September, OpenAI announced that its Safety and Security Committee, which the company introduced in May as it dealt with controversy over security processes, would become an independent board oversight committee. It recently wrapped up its 90-day review evaluating OpenAI’s processes and safeguards and then made recommendations to the board, with the findings also released in a public blog post.

News of the executive departures and board changes also follows a summer of mounting safety concerns and controversies surrounding OpenAI, which along with GoogleMicrosoftMeta and other companies is at the helm of a generative AI arms race — a market that is predicted to top $1 trillion in revenue within a decade — as companies in seemingly every industry rush to add AI-powered chatbots and agents to avoid being left behind by competitors.

In July, OpenAI reassigned Aleksander Madry, one of OpenAI’s top safety executives, to a job focused on AI reasoning instead, sources familiar with the situation confirmed to CNBC at the time.

Madry was OpenAI’s head of preparedness, a team that was “tasked with tracking, evaluating, forecasting, and helping protect against catastrophic risks related to frontier AI models,” according to a bio for Madry on a Princeton University AI initiative website. Madry will still work on core AI safety work in his new role, OpenAI told CNBC at the time.

The decision to reassign Madry came around the same time that Democratic senators sent a letter to OpenAI CEO Sam Altman concerning “questions about how OpenAI is addressing emerging safety concerns.”

The letter, which was viewed by CNBC, also stated, “We seek additional information from OpenAI about the steps that the company is taking to meet its public commitments on safety, how the company is internally evaluating its progress on those commitments, and on the company’s identification and mitigation of cybersecurity threats.”

Microsoft gave up its observer seat on OpenAI’s board in July, stating in a letter viewed by CNBC that it can now step aside because it’s satisfied with the construction of the startup’s board, which had been revamped since the uprising that led to the brief ouster of Altman and threatened Microsoft’s massive investment in the company.

But in June, a group of current and former OpenAI employees published an open letter describing concerns about the artificial intelligence industry’s rapid advancement despite a lack of oversight and an absence of whistleblower protections for those who wish to speak up.

“AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this,” the employees wrote at the time.

Days after the letter was published, a source familiar to the mater confirmed to CNBC that the Federal Trade Commission and the Department of Justice were set to open antitrust investigations into OpenAI, Microsoft and Nvidia, focusing on the companies’ conduct.

FTC Chair Lina Khan has described her agency’s action as a “market inquiry into the investments and partnerships being formed between AI developers and major cloud service providers.”

The current and former employees wrote in the June letter that AI companies have “substantial non-public information” about what their technology can do, the extent of the safety measures they’ve put in place and the risk levels that technology has for different types of harm.

“We also understand the serious risks posed by these technologies,” they wrote, adding the companies “currently have only weak obligations to share some of this information with governments, and none with civil society. We do not think they can all be relied upon to share it voluntarily.”

OpenAI’s Superalignment team, announced last year and disbanded in May, had focused on “scientific and technical breakthroughs to steer and control AI systems much smarter than us.” At the time, OpenAI said it would commit 20% of its computing power to the initiative over four years.

The team was disbanded after its leaders, OpenAI co-founder Ilya Sutskever and Jan Leike, announced their departures from the startup in May. Leike wrote in a post on X that OpenAI’s “safety culture and processes have taken a backseat to shiny products.”

Altman said at the time on X he was sad to see Leike leave and that OpenAI had more work to do. Soon afterward, co-founder Greg Brockman posted a statement attributed to Brockman and the CEO on X, asserting the company has “raised awareness of the risks and opportunities of AGI so that the world can better prepare for it.”

“I joined because I thought OpenAI would be the best place in the world to do this research,” Leike wrote on X at the time. “However, I have been disagreeing with OpenAI leadership about the company’s core priorities for quite some time, until we finally reached a breaking point.”

Leike wrote that he believes much more of the company’s bandwidth should be focused on security, monitoring, preparedness, safety and societal impact.

“These problems are quite hard to get right, and I am concerned we aren’t on a trajectory to get there,” he wrote at the time. “Over the past few months my team has been sailing against the wind. Sometimes we were struggling for [computing resources] and it was getting harder and harder to get this crucial research done.”

Leike added that OpenAI must become a “safety-first AGI company.”

“Building smarter-than-human machines is an inherently dangerous endeavor,” he wrote on X. “OpenAI is shouldering an enormous responsibility on behalf of all of humanity. But over the past years, safety culture and processes have taken a backseat to shiny products.”

Continue Reading

Technology

Cloud software vendors Atlassian, Snowflake and Workday are betting on security startup Veza

Published

on

By

Cloud software vendors Atlassian, Snowflake and Workday are betting on security startup Veza

From left, Veza founders Rob Whitcher, Tarun Thakur and Maohua Lu.

Veza

Tech giants like Google, Amazon, Microsoft and Nvidia have captured headlines in recent years for their massive investments in artificial intelligence startups like OpenAI and Anthropic.

But when it comes to corporate investing by tech companies, cloud software vendors are getting aggressive as well. And in some cases they’re banding together.

Veza, whose software helps companies manage the various internal technologies that employees can access, has just raised $108 million in a financing round that included participation from software vendors Atlassian, Snowflake and Workday.

New Enterprise Associates led the round, which values Veza at just over $800 million, including the fresh capital.

For two years, Snowflake’s managers have used Veza to check who has read and write access, Harsha Kapre, director of the data analytics software company’s venture group told CNBC. It sits alongside a host of other cloud solutions the company uses.

“We have Workday, we have Salesforce — we have all these things,” Kapre said. “What Veza really unlocks for us is understanding who has access and determining who should have access.”

Kapre said that “over-provisioning,” or allowing too many people access to too much stuff, “raises the odds of an attack, because there’s just a lot of stuff that no one is even paying attention to.”

With Veza, administrators can check which employees and automated accounts have authorization to see corporate data, while managing policies for new hires and departures. Managers can approve or reject existing permissions in the software.

Veza says it has built hooks into more than 250 technologies, including Snowflake.

The IPO market is likely to pick up near Labor Day, says FirstMark's Rick Heitzmann

The funding lands at a challenging time for traditional venture firms. Since inflation started soaring in late 2021 and was followed by rising interest rates, startup exits have cooled dramatically, meaning venture firms are struggling to generate returns.

Wall Street was banking on a revival in the initial public offering market with President Donald Trump’s return to the White House, but the president’s sweeping tariff proposals led several companies to delay their offerings.

That all means startup investors have to preserve their cash as well.

In the first quarter, venture firms made 7,551 deals, down from more than 11,000 in the same quarter a year ago, according to a report from researcher PitchBook.

Corporate venture operates differently as the capital comes from the parent company and many investments are strategic, not just about generating financial returns.

Atlassian’s standard agreement asks that portfolio companies disclose each quarter the percentage of a startup’s customers that integrate with Atlassian. Snowflake looks at how much extra product consumption of its own technology occurs as a result of its startup investments, Kapre said, adding that the company has increased its pace of deal-making in the past year.

‘Sleeping industry’

Within the tech startup world, Veza is also in a relatively advantageous spot, because the proliferation of cyberattacks has lifted the importance of next-generation security software.

On the public markets, the First Trust Nasdaq Cybersecurity ETF, which includes CrowdStrike and Palo Alto Networks, is up 3% so far this year, compared with a 10% drop in the Nasdaq.

Veza’s technology runs across a variety of security areas tied to identity and access. In access management, Microsoft is the leader, and Okta is the challenger. Veza isn’t directly competing there, and is instead focused on visibility, an area where other players in and around the space lack technology, said Brian Guthrie, an analyst at Gartner.

Tarun Thakur, Veza’s co-founder and CEO, said his company’s software has become a key part of the ecosystem as other security vendors have started seeing permissions and entitlements as a place to gain broad access to corporate networks.

“We have woken up a sleeping industry,” Thakur, who helped start the company in 2020, said in an interview.

Thakur’s home in Los Gatos, California, doubles as headquarters for the startup, which employs 200 people. It isn’t disclosing revenue figures but says sales more than doubled in the fiscal year that ended in January. Customers include AMD, CrowdStrike and Intuit.

Guthrie said enterprises started recognizing that they needed stronger visibility about two years ago.

“I think it’s because of the number of identities,” he said. Companies realized they had an audit problem or “an account that got compromised,” Guthrie said.

AI agents create a new challenge. Last week Microsoft published a report that advised organizations to figure out the proper ratio of agents to humans.

Veza is building enhancements to enable richer support for agent identities, Thakur said. The new funding will also help Veza expand in the U.S. government and internationally and build more integrations, he said.

Peter Lenke, head of Atlassian’s venture arm, said his company isn’t yet a paying Veza client.

“There’s always potential down the road,” he said. Lenke said he heard about Veza from another investor well before the new round and decided to pursue a stake when the opportunity arose.

Lenke said that startups benefit from Atlassian investments because the company “has a large footprint” inside of enterprises.

“I think there’s a great symbiotic match there,” he said.

Don’t miss these insights from CNBC PRO

Making deals with Menlo Ventures' Matt Murphy

Continue Reading

Technology

IBM pledges $150 billion to boost U.S. tech growth, computer manufacturing

Published

on

By

IBM pledges 0 billion to boost U.S. tech growth, computer manufacturing

Arvind Krishna, chief executive officer of International Business Machines Corp. (IBM), during a Bloomberg Television interview at the World Governments Summit in Dubai, United Arab Emirates, on Tuesday, Feb. 11, 2025.

Christopher Pike | Bloomberg | Getty Images

International Business Machines Corporation on Monday announced it will invest $150 billion in the U.S. over the next five years, including more than $30 billion to advance American manufacturing of its mainframe and quantum computers.

“We have been focused on American jobs and manufacturing since our founding 114 years ago, and with this investment and manufacturing commitment we are ensuring that IBM remains the epicenter of the world’s most advanced computing and AI capabilities,” IBM CEO Arvind Krishna said in a release.   

The company’s announcement comes weeks after President Donald Trump unveiled a far-reaching and aggressive “reciprocal” tariff policy to boost manufacturing in the U.S. As of late April, Trump has exempted chips, as well as smartphonescomputers, and other tech devices and components, from the tariffs.

IBM said its investment will help accelerate America’s role as a global leader in computing and fuel the economy. The company said it operates the “world’s largest fleet of quantum computer systems,” and will continue to build and assemble them in the U.S., according to the release.

Read more CNBC tech news

IBM competitor Nvidia, the chipmaker that has been the primary benefactor of the artificial intelligence boom, announced a similar push earlier this month to produce its NVIDIA AI supercomputers entirely in the U.S. 

Nvidia plans to produce up to $500 billion of AI infrastructure in the U.S. via its manufacturing partnerships over the next four years.

Last week, IBM reported better-than-expected first-quarter results. The company said it generated $14.54 billion in revenue for the period, above the $14.4 billion expected by analysts. IBM’s net income narrowed to $1.06 billion, or $1.12 per share, from $1.61 billion, or $1.72 per share, in the same quarter a year ago.

IBM’s infrastructure division, which includes mainframe computers, posted $2.89 billion in revenue for the quarter, beating expectations of $2.76 billion.

The company announced a new z17 AI mainframe earlier this month.

CNBC’s Jordan Novet contributed to this report.

Continue Reading

Technology

Meta’s AI spending comes into focus amid Trump’s tariff policies

Published

on

By

Meta’s AI spending comes into focus amid Trump’s tariff policies

Meta CEO Mark Zuckerberg looks on before the luncheon on the inauguration day of U.S. President Donald Trump’s second Presidential term in Washington, U.S., Jan. 20, 2025. 

Evelyn Hockstein | Reuters

Mark Zuckerberg’s plan is to make Meta the market leader in artificial intelligence. Investors will want to know how President Donald Trump’s tariffs-heavy trade policies will impact that strategy. 

Those answers could start to come as soon as this week as Meta’s AI strategy takes center stage when the company hosts its first Llama-branded conference for AI developers on Tuesday then reports its latest quarterly earnings the next day.

Already, tech companies are starting to talk about the potential impact they’re bracing for as a result of the Trump tariffs. 

Intel Chief Financial Officer David Zinsner said Thursday during the chip giant’s first-quarter earnings call that U.S. trade policies “have increased the chance of an economic slowdown, with the probability of a recession growing.” Meanwhile, Google CFO Anat Ashkenazi said that day during a first-quarter earnings call that the tech giant remains committed to its $75 billion investment in capital expenditures, or capex, this year, but also acknowledged that the “timing of deliveries and construction schedules” could cause some quarter-to-quarter spending fluctuation. 

For now, analysts expect Meta to follow Alphabet’s lead and remain firm in its plan to spend as much as $65 billion in capex for AI infrastructure this year when it reports earnings Wednesday. Some analysts believe Meta could even raise the figure because AI is a core priority for the company.

“We do not expect META to cut its CapX guidance of $60B-$65B in 2025, for its GenAI infrastructure,  because they see this as an important 10-year investment, we believe,” Needham analysts wrote in a research note published Wednesday. “However, tariffs add risks of upward cost revisions.”

Investors will also be monitoring Meta’s LlamaCon event at its Menlo Park, California, headquarters for any signs that its AI investments are having an immediate business impact. This will be the first time Meta hosts a developer conference specifically for its Llama family of AI models.

“Investors want to see ROI on all these AI investments, and while Meta has shown clear benefits from leveraging AI to improve its products and drive faster revenue growth, it’s been hard to quantify those benefits,” Truist Securities analyst Youssef Squali told CNBC.

Meta in April released a couple of its new Llama 4 models, which Meta Chief Product Officer Chris Cox previously said can help power so-called AI agents that can perform tasks for users via web browsers and other online interfaces.

It’s critical that Meta keep improving Llama to create a major business involving AI agents that companies can use to interact with their customers within apps like Facebook and WhatsApp, William Blair research analyst Ralph Schackart said.

Meta has an early mover advantage at scale in a multi-trillion dollar market,” Schackart said in an email. “We believe Meta is very well positioned to leverage its billions of global users across multiple platforms.”

Meta is unlikely to curb its Llama investment any time soon, but should eventually consider doing so if it fails to generates enough money to justify its costs, said Ken Gawrelski, a Wells Fargo managing director of equity research.

“We do believe that over time Meta needs to continue to evaluate whether Llama needs to be competitive with the leading-edge models,” Gawrelski said. “This is a very expensive proposition and thus far, unlike Google, Meta does not directly monetize its model in any material way.”

Chris Cox, Chief Product Officer at Meta Platforms, speaks during The Wall Street Journal’s WSJ Tech Live Conference in Laguna Beach, California on October 17, 2023. 

Patrick T. Fallon | AFP | Getty Images

Meta AI and the consumer

Analysts are also following the Meta AI digital assistant. That’s because the ChatGPT rival represents the second pillar of Zuckerberg‘s AI strategy. 

Zuckerberg in January said he believes 2025 “is going to be the year when a highly intelligent and personalized AI assistant reaches more than 1 billion people, and I expect Meta AI to be that leading AI assistant.”

In February, CNBC reported that Meta was planning to debut a standalone Meta AI app during the second quarter and test a paid subscription service, in which users could pay monthly fees to access more powerful versions like users can with ChatGPT. 

Although Meta’s enormous user base across its family of apps gives Meta AI an advantage over rivals like ChatGPT in terms of reach, they may not interact with Meta AI in the same way they do with rival chat apps, said Cantor Fitzgerald analyst Deepak Mathivanan.

Gawrelski said that people may not want to use Meta AI within Facebook and Instagram if all they want to do is passively watch the short videos that Meta algorithmically recommends to their feeds.

“This is why a separate Meta AI, where Meta could clearly articulate its use case and value proposition, could be helpful,” Gawrelski said.

A standalone Meta AI app could help the company better market the digital assistant and distinguish it from rivals, said Debra Aho Williamson, founder and chief analyst for Sonata Insights.

“ChatGPT has such wide brand awareness, that it’s become a moat that is soon going to be very hard to overcome,” Williamson said.

Don’t miss these insights from CNBC PRO

The FTC has an uphill battle in its antitrust case against Meta: Former Facebook general counsel

Continue Reading

Trending