The weight-loss drug boom has become one of the internet’s biggest scams
A production line of Wegovy injection pens for the Asian market at the Novo Nordisk A/S pharmaceutical manufacturing facility in Hillerod, Denmark, on Wednesday, Nov. 27, 2024.
Bloomberg | Bloomberg | Getty Images
One interpretation of the law of supply and demand is that when demand outstrips supply, scammers get busy. That’s certainly the case with the super-popular weight-loss drugs from Eli Lilly and Novo Nordisk.
As millions of Americans are prescribed injectable Ozempic and Mounjaro to treat type 2 diabetes, and Wegovy and Zepbound for obesity — and countless more without prescriptions seek them as “vanity drugs” to shed unwanted pounds — the manufacturers can’t keep up production. The GLP-1s, as they’re known, are pricey, too, and insurance often doesn’t cover them, provided consumers can find them.
That confluence of factors has laid the groundwork not only for a confusing online marketplace for compounded versions of the drugs — allowed by the Food and Drug Administration when proprietary ingredients are determined to be in short supply — but a proliferation of nefarious scams offering to sell both brand-name and counterfeit GLP-1s on websites and social media platforms.
Consumers have received Lilly- and Novo-branded GLP-1s from unauthorized sellers, counterfeit versions, completely different medications or nothing at all — other than an expensive rip-off. Most disturbing, Novo told CNBC that as of mid-November, it is aware of 14 deaths and 144 hospitalizations of people who had taken compounded semaglutide, the active pharmaceutical ingredient in Ozempic and Wegovy. It recently asked the FDA to ban the copycat drugs.
Within the past year, cybersecurity experts, consumer advocates, pharma researchers and media investigators have uncovered scores of accounts and content on TikTok, Facebook, Instagram and other social media platforms, as well as numerous websites, where bad actors have been doing business, much of it illegal or at least unethical.
In May, a joint investigation by the nonprofits Digital Citizens Alliance and Coalition for a Safer Web revealed how consumers are flocking to TikTok — which faces an uncertain future after a federal court on Friday upheld a law that would seek to ban the company in the U.S. on Jan. 19 — and other social media platforms and websites to purchase branded and illicit GLP-1s, often without a prescription. According to the report, scammers create accounts promising to sell the drugs for between $200 and $400 for a month’s supply — far below market prices — paid through Zelle, Venmo and PayPal rather than traditional credit cards so as to avoid tracking.
“Scammers take advantage of human emotion and human want, and the emotion and want now is that everybody wants to lose weight,” said Eric Feinberg, vice president of content moderation for the Coalition for a Safer Web. “It’s a perfect audience to use online to take advantage of people psychologically and emotionally.”
A common ruse the investigation exposed was sellers saying the drugs were coming from overseas and then claiming that the order was held up in customs, requiring an additional $300 to $500 payment to release it. The scammers were devious, said Tom Galvin, executive director of Digital Citizens Alliance. “They send a tracking number from a delivery service that shows you where your package is, but the tracking number is BS.” Digital Citizens shelled out just over $3,000 to purchase GLP-1s, and yet the money yielded no deliveries of the drugs.
No-delivery ploys can exact a serious financial toll on victims, but “the more scary ones are where you do get a product and don’t even know whether you can trust [it] or if it’s a valid company,” said Abhishek Karnik, director for threat research and response for cybersecurity firm McAfee.
Phishing for weight-loss drug victims
Tracking activity over the first four months of this year, McAfee’s Threat Research Team uncovered just how prolific weight-loss scams have become across malicious websites, scam emails and texts, posts on social media and online marketplace listings. From January through April, McAfee researchers discovered 449 risky website URLs and 176,871 dangerous phishing attempts centered around Ozempic, Wegovy and semaglutide, an increase of 183% compared to October through December 2023.
Karnik’s team has continued to monitor these criminal activities. “We’ve identified [a total of] 367,000-plus phishing attempts, and between May and August, the number of [risky] URLs we found increased by 135%,” he said.
JAMA Network Open in August published the results of a study by an international group of researchers who searched the global internet to ferret out websites for online pharmacies advertising semaglutide for sale. Among the 317 operations found, more than 42% were illegal, operating without a valid license, selling medications without prescriptions and shipping unregistered and falsified products. Six purchases were made, but only three were delivered.
A recent CNBC investigation explored the murky international world of counterfeit weight-loss drugs. Among its findings, investigators recounted the seizure in the UK last year of hundreds of what appeared to be Ozempic pens, but were in fact insulin pens relabeled as Ozempic. They also discovered from Lilly that its retatrutide, a novel GLP-1 drug still in clinical trials and not FDA-approved, was being marketed to the public.
Counterfeits and diverted drugs — branded GLP-1s sold on the black market — originate from many countries, including India, China, the UK, Mexico and Turkey. One of the destinations where they make their way to the U.S. was New York’s JFK International Airport. According to the U.S. Customs and Border Protection, since January 1, the agency had made more than 198 seizures of products labeled as Ozempic.
In response to this glut of fraudulent activity, social media companies and web operators have employed human monitors and machine technology to identify and shut down online scammers. A TikTok spokesperson, without detailing its various monitoring efforts, referred to the company’s community guidelines. “We strictly prohibit the trade of drugs, and we do not allow attempts to defraud or scam members of our community,” the spokesperson said. “Our advertising policies also prohibit the advertising of weight-loss products, including weight-loss injections and fat-burning pills.”
Despite official policies, however, undeterred violators find workarounds when their accounts are shuttered. They might set up another account with the drug names misspelled, spaces between letters or mash-ups of semaglutide and terzepitide. Many instruct interested buyers to direct message them or send links to Telegram and other dark websites that encrypt content and provide anonymity.
“The social media platforms are the new street corners for drug dealers, and they move from place to place,” Galvin said. “It’s a game of whack-a-mole.”
Bags of counterfeit Novo Nordisk A/S Ozempic and Wegovy, foreground, and other fake drugs at a warehouse operated by the UK’s Medicines and Healthcare Products Regulatory Agency (MHRA) in London, UK, on Monday, Feb. 27, 2024. The UK task force tracks down illegal websites, monitors social media and even carries out raids to stamp out sales of fake “skinny jabs” as both organized crime and unscrupulous lone entrepreneurs look to capitalize on the weight-loss frenzy.
Bloomberg | Bloomberg | Getty Images
For this article, CNBC found more than a dozen TikTok accounts that appeared to be selling GLP-1s in violation of its policies, including @ozempic_weightloss, @sema.irel and @semaglutideandtr. Soon after relaying the information to TikTok, we were told that all had been removed, except one, which was not in violation.
The widespread compounding of GLP-1s is another contributor to the dodgy marketplace for the drugs. In April and December of 2022, respectively, the FDA determined that semaglutide and tirzepatide were in short supply, opening the floodgates for compounding pharmacies and outsourcing facilities to manufacture, distribute and market copies, typically sold through telehealth companies, medical spas and wellness centers.
Compounded GLP-1s, unlike Lilly’s and Novo’s brands, are not FDA-approved, which means they do not undergo the agency’s review for safety, effectiveness and quality before they’re marketed. Instead, the FDA and state boards of pharmacy register, license and inspect compounding facilities and ingredients. And while some compounders meet regulatory requirements, such as Henry Meds, Noom Med, Ro and Hims & Hers Health, many others don’t.
Publicly traded Hims & Hers launched its gender-focused telehealth platform in 2017, adding compounded semaglutide to its weight-loss program this past May. “We waited until we were able to find the right compounding partner,” said Dr. Patrick Carroll, the company’s chief medial officer. Besides that partner, BPI Labs, Hims & Hers acquired another, MetasourceRx, in September. The company also sells branded Ozempic and next year will offer liraglutide, the first generic GLP-1.
FDA scrutiny
In the meantime, the FDA is investigating the bad actors in the compounding world. “Purchasing prescription drugs from unregulated, unlicensed sources without a prescription is risky,” a spokesperson for the agency told CNBC. “We urge consumers to be vigilant and to utilize tips tools from the FDA’s BeSafeRx campaign to help them safely buy drugs online.”
In May, the KFF Health Tracking Poll found that about one in eight adults (12%) said they had taken a GLP-1 drug, with about half, or 21 million, actively using the medications. Nearly 80% purchased the drugs or a prescription for them — at a cost between $936 to $1,349 per month before insurance coverage, rebates or coupons — from a primary care doctor or a specialist, according to the survey. Fewer reported getting them from an online provider or website (11%), a medical spa or aesthetic medical center (10%), or from somewhere else (2%). But that doesn’t count the inestimable number of individuals who have obtained GLP-1s without prescriptions through unregulated online channels and illicit online compounding pharmacies, many operating overseas.
While social media companies police illegal sellers of GLP-1s, hundreds of influencers are touting the drugs and their journeys using them across the platforms with impunity, according to a Fast Company report. Many influencers are recruited and paid by telehealth companies.
Meanwhile, household names have been increasingly speaking out about their personal use of these drugs, which increases familiarity and curiosity among the public. In October, People profiled 64 celebrities — including Kathy Bates, Elon Musk, Oprah Winfrey, Andy Cohen, Billie Jean King and Rob Lowe — who have talked about their weight-loss drug experiences, mostly on social media.
Currently, Lilly’s and Novo’s GLP-1s are prescribed only for type 2 diabetes and obesity. But as researchers find additional conditions that can be treated with the drugs — including cardiovascular disease, kidney disease, dementia and addiction, and most recently even knee pain — prescriptions will increase exponentially.
In September, an article in the Annals of Pharmacotherapy warned against manufacturers that use a legal loophole to sell vials containing semaglutide and tirzepatide to consumers without a prescription by stating that the drugs are for “research purposes only” and/or “not for human consumption.” The authors conducted an internet search for such scofflaws, uncovering 40 websites selling what were labeled as “peptides” to consumers.
The FDA has sent warning letters to a handful, including Miami-based US Chem Labs in February, citing several violations and requesting action within 15 days. As of Dec. 6, CNBC found that the company still listed compounded semaglutide as available on its website. US Chem Labs could not be reached by phone and an email request for comment was not returned by press time.
The authors of the Annals of Pharmacotherapy article also identified three companies that were advertising GLP-1s on Facebook, owned by Meta. “Our policies prohibit content that defrauds people by promoting false or misleading health claims, including those related to weight loss, and we remove this kind of content when we become aware of it,” a Meta spokesperson told CNBC. CNBC subsequently sent Meta the names of the three companies, and several days later their Facebook pages were removed.
Eli Lilly, Novo Nordisk battle with copycat drugs
Workers walk past manufacturing equipment at Eli Lilly & Co. manufacturing plant in Kinsale, Ireland, on Sept. 12, 2024. Lilly has been bulking up its production capacity since 2020, investing more than $17 billion into developing new plants and expanding existing facilities for the weight-loss and diabetes drugs that are expected to become some of the best-selling medicines of all time.
Bloomberg | Bloomberg | Getty Images
Lilly and Novo are in a quandary regarding compounders. The copycats have filled a void while the branded GLP-1s are in shortage, attracting patients who can’t access or afford them.
But now the manufacturers want their domains to themselves. Lilly has sent cease-and-desist letters to numerous compounding sellers, and both companies have filed lawsuits against numerous compounding pharmacies, alleging trademark infringement and deceptive marketing.
On October 2, the FDA declared that Lilly’s tirzepatide was no longer in short supply, ostensibly putting compounders of that ingredient out of business. Two weeks later, though, after a public outcry from compounders’ patients and a federal lawsuit brought by compounding pharmacies, the FDA backtracked, saying it would reevaluate whether the drug is available and make a decision in mid-November.
Yet, on November 22, the FDA said it was still assessing the situation and agreed to not take action against compounders of tirzepatide until December 19, unless the agency makes an earlier decision.
Novo’s semaglutide is still listed as “currently in shortage” by the FDA, although the agency also lists Ozempic and Wegovy as “available.” A Novo Nordisk spokesperson told CNBC, “It’s important to note that availability doesn’t always mean immediate accessibility at every pharmacy. Patients may experience variability at specific locations, regardless of whether a drug is in shortage.”
Lilly and Novo have advocated for broadening insurance coverage for the drugs, and the Biden administration recently proposed that Medicare and Medicaid extend their coverage for obesity medications. Although that plan could be scuttled by the incoming Trump administration. Robert F. Kennedy, Jr., Trump’s nominee to head the Department of Health and Human Services, has suggested that obesity should be tackled through healthy eating, not drugs.
The obesity drug market volatility has shown up in recent earnings. In its third-quarter report on October 30, Lilly fell short of profit and revenue expectations, partly due to disappointing sales of its GLP-1s, even as demand for them continued to soar. A week later, Novo reported third-quarter earnings in line with expectations, strengthened by robust sales of Ozempic and Wegovy. Nonetheless, the Danish company narrowed its 2024 full-year growth guidance, reflecting, according to a statement from the company, “expected continued periodic supply constraints and related drug shortage notifications.”
Both pharma giants continue to invest billions to increase production facilities and capacity. This week, Lilly said it was investing $3 billion to increase obesity drug production at a Wisconsin plant.
Regardless, demand for GLP-1s — no matter if they’re branded, compounded or counterfeit or where they’re purchased from — is certain to keep growing. That will put more pressure on social media platforms and web operators to guard against scams.
Galvin suggested that the companies need to work together to identify scammers as they navigate between platforms to avoid detection. “Too many platforms look at this as a PR problem and not an internet safety problem,” he said. “If they were collaborating with each other to identify the bad actors and shared that information, people would find a lot less of them.”
Technology
AI research takes a backseat to profits as Silicon Valley prioritizes products over safety, experts say
Sam Altman, co-founder and CEO of OpenAI and co-founder of Tools for Humanity, participates remotely in a discussion on the sidelines of the IMF/World Bank Spring Meetings in Washington, D.C., April 24, 2025.
Brendan Smialowski | AFP | Getty Images
Not long ago, Silicon Valley was where the world’s leading artificial intelligence experts went to perform cutting-edge research.
Meta, Google and OpenAI opened their wallets for top talent, giving researchers staff, computing power and plenty of flexibility. With the support of their employers, the researchers published high-quality academic papers, openly sharing their breakthroughs with peers in academia and at rival companies.
But that era has ended. Now, experts say, AI is all about the product.
Since OpenAI released ChatGPT in late 2022, the tech industry has shifted its focus to building consumer-ready AI services, in many cases prioritizing commercialization over research, AI researchers and experts in the field told CNBC. The profit potential is massive — some analysts predict $1 trillion in annual revenue by 2028. The prospective repercussions terrify the corner of the AI universe concerned about safety, industry experts said, particularly as leading players pursue artificial general intelligence, or AGI, which is technology that rivals or exceeds human intelligence.
In the race to stay competitive, tech companies are taking an increasing number of shortcuts when it comes to the rigorous safety testing of their AI models before they are released to the public, industry experts told CNBC.
James White, chief technology officer at cybersecurity startup CalypsoAI, said newer models are sacrificing security for quality, that is, better responses by the AI chatbots. That means they’re less likely to reject malicious kinds of prompts that could cause them to reveal ways to build bombs or sensitive information that hackers could exploit, White said.
“The models are getting better, but they’re also more likely to be good at bad stuff,” said White, whose company performs safety and security audits of popular models from Meta, Google, OpenAI and other companies. “It’s easier to trick them to do bad stuff.”
The changes are readily apparent at Meta and Alphabet, which have deprioritized their AI research labs, experts say. At Facebook’s parent company, the Fundamental Artificial Intelligence Research, or FAIR, unit has been sidelined by Meta GenAI, according to current and former employees. And at Alphabet, the research group Google Brain is now part of DeepMind, the division that leads development of AI products at the tech company.
CNBC spoke with more than a dozen AI professionals in Silicon Valley who collectively tell the story of a dramatic shift in the industry away from research and toward revenue-generating products. Some are former employees at the companies with direct knowledge of what they say is the prioritization of building new AI products at the expense of research and safety checks. They say employees face intensifying development timelines, reinforcing the idea that they can’t afford to fall behind when it comes to getting new models and products to market. Some of the people asked not to be named because they weren’t authorized to speak publicly on the matter.
Mark Zuckerberg, CEO of Meta Platforms, during the Meta Connect event in Menlo Park, California, on Sept. 25, 2024.
David Paul Morris | Bloomberg | Getty Images
Meta’s AI evolution
When Joelle Pineau, a Meta vice president and the head of the company’s FAIR division, announced in April that she would be leaving her post, many former employees said they weren’t surprised. They said they viewed it as solidifying the company’s move away from AI research and toward prioritizing developing practical products.
“Today, as the world undergoes significant change, as the race for AI accelerates, and as Meta prepares for its next chapter, it is time to create space for others to pursue the work,” Pineau wrote on LinkedIn, adding that she will formally leave the company May 30.
Pineau began leading FAIR in 2023. The unit was established a decade earlier to work on difficult computer science problems typically tackled by academia. Yann LeCun, one of the godfathers of modern AI, initially oversaw the project, and instilled the research methodologies he learned from his time at the pioneering AT&T Bell Laboratories, according to several former employees at Meta. Small research teams could work on a variety of bleeding-edge projects that may or may not pan out.
The shift began when Meta laid off 21,000 employees, or nearly a quarter of its workforce, starting in late 2022. CEO Mark Zuckerberg kicked off 2023 by calling it the “year of efficiency.” FAIR researchers, as part of the cost-cutting measures, were directed to work more closely with product teams, several former employees said.
Two months before Pineau’s announcement, one of FAIR’s directors, Kim Hazelwood, left the company, two people familiar with the matter said. Hazelwood helped oversee FAIR’s NextSys unit, which manages computing resources for FAIR researchers. Her role was eliminated as part of Meta’s plan to cut 5% of its workforce, the people said.
Joelle Pineau of Meta speaks at the Advancing Sustainable Development through Safe, Secure, and Trustworthy AI event at Grand Central Terminal in New York, Sept. 23, 2024.
Bryan R. Smith | Via Reuters
OpenAI’s 2022 launch of ChatGPT caught Meta off guard, creating a sense of urgency to pour more resources into large language models, or LLMs, that were captivating the tech industry, the people said.
In 2023, Meta began heavily pushing its freely available and open-source Llama family of AI models to compete with OpenAI, Google and others.
With Zuckerberg and other executives convinced that LLMs were game-changing technologies, management had less incentive to let FAIR researchers work on far-flung projects, several former employees said. That meant deprioritizing research that could be viewed as having no impact on Meta’s core business, such as FAIR’s previous health care-related research into using AI to improve drug therapies.
Since 2024, Meta Chief Product Officer Chris Cox has been overseeing FAIR as a way to bridge the gap between research and the product-focused GenAI group, people familiar with the matter said. The GenAI unit oversees the Llama family of AI models and the Meta AI digital assistant, the two most important pillars of Meta’s AI strategy.
Under Cox, the GenAI unit has been siphoning more computing resources and team members from FAIR due to its elevated status at Meta, the people said. Many researchers have transferred to GenAI or left the company entirely to launch their own research-focused startups or join rivals, several of the former employees said.
While Zuckerberg has some internal support for pushing the GenAI group to rapidly develop real-world products, there’s also concern among some staffers that Meta is now less able to develop industry-leading breakthroughs that can be derived from experimental work, former employees said. That leaves Meta to chase its rivals.
A high-profile example landed in January, when Chinese lab DeepSeek released its R1 model, catching Meta off guard. The startup claimed it was able to develop a model as capable as its American counterparts but with training at a fraction of the cost.
Meta quickly implemented some of DeepSeek’s innovative techniques for its Llama 4 family of AI models that were released in April, former employees said. The AI research community had a mixed reaction to the smaller versions of Llama 4, but Meta said the biggest and most powerful Llama 4 variant is still being trained.
The company in April also released security and safety tools for developers to use when building apps with Meta’s Llama 4 AI models. These tools help mitigate the chances of Llama 4 unintentionally leaking sensitive information or producing harmful content, Meta said.
“Our commitment to FAIR remains strong,” a Meta spokesperson told CNBC. “Our strategy and plans will not change as a result of recent developments.”
In a statement to CNBC, Pineau said she is enthusiastic about Meta’s overall AI work and strategy.
“There continues to be strong support for exploratory research and FAIR as a distinct organization in Meta,” Pineau said. “The time was simply right for me personally to re-focus my energy before jumping into a new adventure.”
Meta on Thursday named FAIR co-founder Rob Fergus as Pineau’s replacement. Fergus will return to the company to serve as a director at Meta and head of FAIR, according to his LinkedIn profile. He was most recently a research director at Google DeepMind.
“Meta’s commitment to FAIR and long term research remains unwavering,” Fergus said in a LinkedIn post. “We’re working towards building human-level experiences that transform the way we interact with technology and are dedicated to leading and advancing AI research.”
Demis Hassabis, co-founder and CEO of Google DeepMind, attends the Artificial Intelligence Action Summit at the Grand Palais in Paris, Feb. 10, 2025.
Benoit Tessier | Reuters
Google ‘can’t keep building nanny products’
Google released its latest and most powerful AI model, Gemini 2.5, in March. The company described it as “our most intelligent AI model,” and wrote in a March 25 blog post that its new models are “capable of reasoning through their thoughts before responding, resulting in enhanced performance and improved accuracy.”
For weeks, Gemini 2.5 was missing a model card, meaning Google did not share information about how the AI model worked or its limitations and potential dangers upon its release.
Model cards are a common tool for AI transparency.
A Google website compares model cards to food nutrition labels: They outline “the key facts about a model in a clear, digestible format,” the website says.
“By making this information easy to access, model cards support responsible AI development and the adoption of robust, industry-wide standards for broad transparency and evaluation practices,” the website says.
Google wrote in an April 2 blog post that it evaluates its “most advanced models, such as Gemini, for potential dangerous capabilities prior to their release.” Google later updated the blog to remove the words “prior to their release.”
Without a model card for Gemini 2.5, the public had no way of knowing which safety evaluations were conducted or whether DeepMind checked for dangerous capabilities at all.
In response to CNBC’s inquiry on April 2 about Gemini 2.5’s missing model card, a Google spokesperson said that a “tech report with additional safety information and model cards are forthcoming.” Google published an incomplete model card on April 16 and updated it on April 28, more than a month after the AI model’s release, to include information about Gemini 2.5’s “dangerous capability evaluations.”
Those assessments are important for gauging the safety of a model — whether people can use the models to learn how to build chemical or nuclear weapons or hack into important systems. These checks also determine whether a model is capable of autonomously replicating itself, which could lead to a company losing control of it. Running tests for those capabilities requires more time and resources than simple, automated safety evaluations, according to industry experts.
Google co-founder Sergey Brin
Kelly Sullivan | Getty Images Entertainment | Getty Images
The Financial Times in March reported that Google DeepMind CEO Demis Hassabis had installed a more rigorous vetting process for internal research papers to be published. The clampdown at Google is particularly notable because the company’s “Transformers” technology gained recognition across Silicon Valley through that type of shared research. Transformers were critical to OpenAI’s development of ChatGPT and the rise of generative AI.
Google co-founder Sergey Brin told staffers at DeepMind and Gemini in February that competition has accelerated and “the final race to AGI is afoot,” according to a memo viewed by CNBC. “We have all the ingredients to win this race but we are going to have to turbocharge our efforts,” he said in the memo.
Brin said in the memo that Google has to speed up the process of testing AI models, as the company needs “lots of ideas that we can test quickly.”
“We need real wins that scale,” Brin wrote.
In his memo, Brin also wrote that the company’s methods have “a habit of minor tweaking and overfitting” products for evaluations and “sniping” the products at checkpoints. He said employees need to build “capable products” and to “trust our users” more.
“We can’t keep building nanny products,” Brin wrote. “Our products are overrun with filters and punts of various kinds.”
A Google spokesperson told CNBC that the company has always been committed to advancing AI responsibly.
“We continue to do that through the safe development and deployment of our technology, and research contributions to the broader ecosystem,” the spokesperson said.
Sam Altman, CEO of OpenAI, is seen through glass during an event on the sidelines of the Artificial Intelligence Action Summit in Paris, Feb. 11, 2025.
Aurelien Morissard | Via Reuters
OpenAI’s rush through safety testing
The debate of product versus research is at the center of OpenAI’s existence. The company was founded as a nonprofit research lab in 2015 and is now in the midst of a contentious effort to transform into a for-profit entity.
That’s the direction co-founder and CEO Sam Altman has been pushing toward for years. On May 5, though, OpenAI bowed to pressure from civic leaders and former employees, announcing that its nonprofit would retain control of the company even as it restructures into a public benefit corporation.
Nisan Stiennon worked at OpenAI from 2018 to 2020 and was among a group of former employees urging California and Delaware not to approve OpenAI’s restructuring effort. “OpenAI may one day build technology that could get us all killed,” Stiennon wrote in a statement in April. “It is to OpenAI’s credit that it’s controlled by a nonprofit with a duty to humanity.”
But even with the nonprofit maintaining control and majority ownership, OpenAI is speedily working to commercialize products as competition heats up in generative AI. And it may have rushed the rollout of its o1 reasoning model last year, according to some portions of its model card.
Results of the model’s “preparedness evaluations,” the tests OpenAI runs to assess an AI model’s dangerous capabilities and other risks, were based on earlier versions of o1. They had not been run on the final version of the model, according to its model card, which is publicly available.
Johannes Heidecke, OpenAI’s head of safety systems, told CNBC in an interview that the company ran its preparedness evaluations on near-final versions of the o1 model. Minor variations to the model that took place after those tests wouldn’t have contributed to significant jumps in its intelligence or reasoning and thus wouldn’t require additional evaluations, he said. Still, Heidecke acknowledged that OpenAI missed an opportunity to more clearly explain the difference.
OpenAI’s newest reasoning model, o3, released in April, seems to hallucinate more than twice as often as o1, according to the model card. When an AI model hallucinates, it produces falsehoods or illogical information.
OpenAI has also been criticized for reportedly slashing safety testing times from months to days and for omitting the requirement to safety test fine-tuned models in its latest “Preparedness Framework.”
Heidecke said OpenAI has decreased the time needed for safety testing because the company has improved its testing effectiveness and efficiency. A company spokesperson said OpenAI has allocated more AI infrastructure and personnel to its safety testing, and has increased resources for paying experts and growing its network of external testers.
In April, the company shipped GPT-4.1, one of its new models, without a safety report, as the model was not designated by OpenAI as a “frontier model,” which is a term used by the tech industry to refer to a bleeding-edge, large-scale AI model.
But one of those small revisions caused a big wave in April. Within days of updating its GPT-4o model, OpenAI rolled back the changes after screenshots of overly flattering responses to ChatGPT users went viral online. OpenAI said in a blog post explaining its decision that those types of responses to user inquiries “raise safety concerns — including around issues like mental health, emotional over-reliance, or risky behavior.”
OpenAI said in the blogpost that it opted to release the model even after some expert testers flagged that its behavior “‘felt’ slightly off.”
“In the end, we decided to launch the model due to the positive signals from the users who tried out the model. Unfortunately, this was the wrong call,” OpenAI wrote. “Looking back, the qualitative assessments were hinting at something important, and we should’ve paid closer attention. They were picking up on a blind spot in our other evals and metrics.”
Metr, a company OpenAI partners with to test and evaluate its models for safety, said in a recent blog post that it was given less time to test the o3 and o4-mini models than predecessors.
“Limitations in this evaluation prevent us from making robust capability assessments,” Metr wrote, adding that the tests it did were “conducted in a relatively short time.”
Metr also wrote that it had insufficient access to data that would be important in determining the potential dangers of the two models.
The company said it wasn’t able to access the OpenAI models’ internal reasoning, which is “likely to contain important information for interpreting our results.” However, Metr said, “OpenAI shared helpful information on some of their own evaluation results.”
OpenAI’s spokesperson said the company is piloting secure ways of sharing chains of thought for Metr’s research as well as for other third-party organizations.
Steven Adler, a former safety researcher at OpenAI, told CNBC that safety testing a model before it’s rolled out is no longer enough to safeguard against potential dangers.
“You need to be vigilant before and during training to reduce the chance of creating a very capable, misaligned model in the first place,” Adler said.
He warned that companies such as OpenAI are backed into a corner when they create capable but misaligned models with goals that are different from the ones they intended to build.
“Unfortunately, we don’t yet have strong scientific knowledge for fixing these models — just ways of papering over the behavior,” Adler said.
WATCH: OpenAI closes $40 billion funding round, largest private tech deal on record
Omar Marques | Sopa Images | Lightrocket | Getty Images
Shares of stock brokerage platform eToro popped in their Nasdaq debut on Wednesday after the company raised almost $310 million in its initial public offering.
The stock opened at $69.69, or 34% above its IPO, pushing its market cap to $5.6 billion. Shares were last up more than 40%.
The Israel-based company sold nearly six million shares at $52 each, above the expected range of $46 to $50. Almost six million additional shares were sold by existing investors. At the IPO price, the company was valued at roughly $4.2 billion.
Wall Street is looking to the Robinhood competitor for signs of renewed interest in IPOs after an extended drought. Many investors saw President Donald Trump’s return to the White House as a catalyst before tariff concerns led companies to delay their plans.
Etoro isn’t the only company attempting to test the waters. Fintech company Chime filed its prospectus with the U.S. Securities and Exchange Commission on Tuesday, while digital physical therapy company Hinge Health kickstarted its IPO roadshow, and said in a filing it aims to raise up to $437 million in its offering.
EToro had previously filed to go public in 2021 through a merger with a special purpose acquisition company, or SPAC, that would have valued it at more than $10 billion. It shelved those plans in 2022 as equity markets nosedived, but remained focused on an eventual IPO.
EToro was founded in 2007 by brothers Yoni and Ronen Assia and David Ring. The company makes money through trading-related fees and nontrading activities such as withdrawals. Net income increased almost thirteenfold last year to $192.4 million from $15.3 million in 2023.
The company has steadily built a growing business in cryptocurrencies. Revenue from crypto assets more than tripled to upward of $12 million in 2024, and one-quarter of its net trading contribution stemmed from crypto last year. That is up from 10% in 2023.
EToro said that for the first quarter, it expects crypto assets to account for 37% of its commission from trading activities, down from 43% a year earlier.
Spark Capital is the company’s biggest outside investor, with 14% control after the offering, followed by BRM Group at 8.7%. CEO Yoni Assia controls 9.3%.
-
Sports3 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoStory injured on diving stop, exits Red Sox game
-
Sports2 years agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike
-
Business3 years agoBank of England’s extraordinary response to government policy is almost unthinkable | Ed Conway