Connect with us

Published

on

Yoshua Bengio (L) and Max Tegmark (R) discuss the development of artificial general intelligence during a live podcast recording of CNBC’s “Beyond The Valley” in Davos, Switzerland in January 2025.

CNBC

Artificial general intelligence built like “agents” could prove dangerous as its creators might lose control of the system, two of of the world’s most prominent AI scientists told CNBC.

In the latest episode of CNBC’s “Beyond The Valley” podcast released on Tuesday, Max Tegmark, a professor at the Massachusetts Institute of Technology and the President of the Future of Life Institute, and Yoshua Bengio, dubbed one of the “godfathers of AI” and a professor at the Université de Montréal, spoke about their concerns about artificial general intelligence, or AGI. The term broadly refers to AI systems that are smarter than humans.

Their fears stem from the world’s biggest firms now talking about “AI agents” or “agentic AI” — which companies claim will allow AI chatbots to act like assistants or agents and assist in work and everyday life. Industry estimates vary on when AGI will come into existence.

With that concept comes the idea that AI systems could have some “agency” and thoughts of their own, according to Bengio.

“Researchers in AI have been inspired by human intelligence to build machine intelligence, and, in humans, there’s a mix of both the ability to understand the world like pure intelligence and the agentic behavior, meaning … to use your knowledge to achieve goals,” Bengio told CNBC’s “Beyond The Valley.”

“Right now, this is how we’re building AGI: we are trying to make them agents that understand a lot about the world, and then can act accordingly. But this is actually a very dangerous proposition.”

Bengio added that pursuing this approach would be like “creating a new species or a new intelligent entity on this planet” and “not knowing if they’re going to behave in ways that agree with our needs.”

“So instead, we can consider, what are the scenarios in which things go badly and they all rely on agency? In other words, it is because the AI has its own goals that we could be in trouble.”

The idea of self-preservation could also kick in, as AI gets even smarter, Bengio said.

“Do we want to be in competition with entities that are smarter than us? It’s not a very reassuring gamble, right? So we have to understand how self-preservation can emerge as a goal in AI.”

AI tools the key

For MIT’s Tegmark, the key lies in so-called “tool AI” — systems that are created for a specific, narrowly-defined purpose, but that don’t have to be agents.

Tegmark said a tool AI could be a system that tells you how to cure cancer, or something that possesses “some agency” like a self-driving car “where you can prove or get some really high, really reliable guarantees that you’re still going to be able to control it.”

“I think, on an optimistic note here, we can have almost everything that we’re excited about with AI … if we simply insist on having some basic safety standards before people can sell powerful AI systems,” Tegmark said.

“They have to demonstrate that we can keep them under control. Then the industry will innovate rapidly to figure out how to do that better.”

Tegmark’s Future of Life Institute in 2023 called for a pause to the development of AI systems that can compete with human-level intelligence. While that has not happened, Tegmark said people are talking about the topic, and now it is time to take action to figure out how to put guardrails in place to control AGI.

“So at least now a lot of people are talking the talk. We have to see if we can get them to walk the walk,” Tegmark told CNBC’s “Beyond The Valley.”

“It’s clearly insane for us humans to build something way smarter than us before we figured out how to control it.”

There are several views on when AGI will arrive, partly driven by varying definitions.

OpenAI CEO Sam Altman said his company knows how to build AGI and said it will arrive sooner than people think, though he downplayed the impact of the technology.

“My guess is we will hit AGI sooner than most people in the world think and it will matter much less,” Altman said in December.

Continue Reading

Technology

Salesforce pledges to invest $1 billion in Singapore over five years in AI push

Published

on

By

Salesforce pledges to invest  billion in Singapore over five years in AI push

Marc Benioff, Chairman & CEO of Salesforce, speaking on CNBC’s Squawk Box outside the World Economic Forum in Davos, Switzerland on Jan. 22nd, 2025.

Gerry Miller | CNBC

Salesforce on Wednesday announced plans to invest $1 billion in Singapore over the next five years.

The cloud software giant said the investment is designed to accelerate the country’s digital transformation and the adoption of Salesforce’s flagship AI offering Agentforce.

Salesforce is among the many technology companies hoping to boost revenue with generative AI features.

The company launched the newest version of Agentforce last month. It has previously described the system — which it says can tackle sophisticated questions in Salesforce’s Slack communications app, based on all available data — as the first digital AI platform for enterprises.

Salesforce CEO Marc Benioff is scheduled to speak at CNBC’s CONVERGE LIVE at around 9:25 a.m. Singapore time (9:25 p.m. ET) on Wednesday.

“We are in an incredible new era of digital labor where every business will be transformed by autonomous agents that augment the work of humans, revolutionizing productivity and enabling every company to scale without limits,” Benioff said in a statement.

“Singapore is at the forefront of this shift, and as the world’s largest provider of digital labor through our Agentforce platform,” he added.

Salesforce said Agentforce can help Singapore to “rapidly expand” its labor force in several key service and public sector roles at a time when the country is grappling with an aging population and declining birth rates.

Jermaine Loy, managing director of the Singapore Economic Development Board, welcomed Salesforce’s investment, saying it will help to boost the country’s efforts “to build a vibrant hub for AI innovation.”

— CNBC’s Jordan Novet contributed to this report.

Continue Reading

Technology

Reddit rallies after three-day slump as analyst calls sell-off ‘excessive’

Published

on

By

Reddit rallies after three-day slump as analyst calls sell-off 'excessive'

Reddit CEO Steve Huffman stands on the floor of the New York Stock Exchange (NYSE) after ringing a bell on the floor setting the share price at $47 in its initial public offering (IPO) on March 21, 2024 in New York City.

Spencer Platt | Getty Images News | Getty Images

Reddit shares rose more than 10% on Tuesday, reversing a three-day slump that coincided with a broader decline among technology companies.

Despite Tuesday’s gains, Reddit shares are still roughly 30% below the close on Wednesday.

Reddit’s stock market upswing was likely bolstered by a Loop Capital analyst note published Tuesday that reiterated a buy rating and characterized the company’s shares as “extremely attractive.” The analyst note said that Reddit’s 50% drop on Wall Street in the past month “is excessive,” and that the social media company “has the biggest upside potential relative to Street estimates in our coverage universe.”

The company’s shares dropped more than 15% in February after the company reported weaker-than-expected fourth-quarter user numbers as a result of a Google search change that temporarily hurt its search-derived traffic. Although Reddit said at the time that it had recovered from the algorithmic shift, the user number miss spooked investors.

Reddit’s shares have since spiraled downward along with other tech companies like Apple, Nvidia and Tesla off of concerns related to President Donald Trump‘s tariffs and growing fears of a recession. The seven most valuable tech companies lost more than $750 billion in market value on Monday with Nasdaq experiencing its biggest decline since 2022.

Loop Capital managing director Alan Gould acknowledged in the note that investors are operating in a “risk-off market environment,” but he contended that Reddit “has been one of the top performing stocks over the past year,” aside from its most recent dip.

“RDDT wildly exceeded ours and Street estimates for 2024, which explains why the stock increased almost 7-fold from a $34 IPO price to a peak of $230 in less than a year,” Gould wrote, noting Reddit’s growing revenue and improved advertising tools, among other positive developments.

Reddit’s fourth-quarter sales grew 71% year over year to $428 million, which represents the fastest growth rate for any quarter since 2022.

“In our view, RDDT deserves the revaluation it had experiencing based on the growth it has shown in the recent earnings reports and future projected growth driven by the ability to narrow the ARPU gap, and data licensing possibilities,” Gould wrote.

Don’t miss these insights from CNBC PRO

Market is suggesting tech beats aren't sustainable, says T. Rowe Price's Tony Wang

Continue Reading

Technology

Waymo expands its robotaxi service again, this time to parts of Silicon Valley

Published

on

By

Waymo expands its robotaxi service again, this time to parts of Silicon Valley

Waymo self-driving cars with roof-mounted sensor arrays traveling near palm trees and modern buildings along the Embarcadero, San Francisco, California, February 21, 2025. 

Smith Collection/gado | Archive Photos | Getty Images

Waymo on Tuesday announced it is expanding its service to include another 27 square miles of coverage around the San Francisco Bay Area.

With the expansion, Waymo will now take passengers around Mountain View, Los Altos, Palo Alto and parts of Sunnyvale, California. The Alphabet-owned company opened its robotaxi service to the general public in San Francisco in June.

Waymo will initially limit the availability of its Silicon Valley service to users of the Waymo One app who are residents with ZIP codes in the area, the company said. Waymo plans to serve more riders across the region over time. The fleet of vehicles that will be in use in the new coverage areas are fully electric Jaguar I-Pace vehicles with Waymo’s fifth generation of self-driving sensors, software and other technology.

“Opening our fully autonomous ride-hailing service in Silicon Valley marks a special milestone in our Bay Area journey,” Waymo product chief Saswat Panigrahi said in a statement. “This is where Waymo began and where we’re headquartered.”

Waymo expanded its San Francisco Bay Area robotaxi service last summer into Daly City, Broadmoor and Colma. Its robotaxis do not yet carry passengers to San Francisco International Airport.

A spokesperson told CNBC that Waymo is in “active discussions with SFO,” and added that the company is “working to connect” Silicon Valley and San Francisco to “provide seamless autonomous rides across more of the Bay Area in the future.”

Waymo also recently launched a commercial robotaxi service in Austin, Texas, just in time for the city’s annual South by Southwest festival.

While would-be competitors including Elon Musk‘s automaker Tesla, and Amazon-owned Zoox, are continuing their own robotaxi testing and development, Waymo has pulled far ahead of self-driving companies in the U.S. 

Before Tuesday’s expansion, Waymo said it was serving more than 200,000 paid trips per week across San Francisco, Los Angeles and Phoenix.

Alphabet doesn’t disclose financial results for the autonomous vehicle business, but Waymo is part of its “Other Bets.” That business unit generated $400 million in the fourth quarter of 2024 and incurred operating losses of $1.17 billion, according to the company’s most recent financial filing.

Don’t miss these insights from CNBC PRO

The rise of Phoenix as a major tech hub with chips, autonomous cars and drones

Continue Reading

Trending