People using their mobile phones outside the offices of Meta, the parent company of Facebook and Instagram, in King’s Cross, London.
Joshua Bratt | Pa Images | Getty Images
Lauren Wagner knows a lot about disinformation. Heading into the 2020 U.S. presidential election, she worked at Facebook, focusing on information integrity and overseeing products designed to make sure content was moderated and fact-checked.
She can’t believe what’s she’s seeing now. Since war erupted last month between Israel and Hamas, the constant deluge of misinformation and violent content spreading across the internet is hard for her to comprehend. Wagner left Facebook parent Meta last year, and her work in trust and safety feels like it was from a prior era.
“When you’re in a situation where there’s such a large volume of visual content, how do you even start managing that when it’s like long video clips and there’s multiple points of view?” Wagner said. “This idea of live-streaming terrorism, essentially at such a deep and in-depth scale, I don’t know how you manage that.”
The problem is even more pronounced because Meta, Google parent Alphabet, and X, formerly Twitter, have all eliminated jobs tied to content moderation and trust and safety as part of broader cost-cutting measures that began late last year and continued through 2023. Now, as people post and share out-of-context videos of previous wars, fabricated audio in news clips, and graphic videos of terrorist acts, the world’s most trafficked websites are struggling to keep up, experts have noted.
As the founder of a new venture capital firm, Radium Ventures, Wagner is in the midst of raising her first fund dedicated solely to startup founders working on trust and safety technologies. She said many more platforms that think they are “fairly innocuous” are seeing the need to act.
“Hopefully this is shining a light on the fact that if you house user-generated content, there’s an opportunity for misinformation, for charged information or potentially damaging information to spread,” Wagner said.
In addition to the traditional social networks, the highly polarized nature of the Israel-Hamas war affects internet platforms that weren’t typically known for hosting political discussions but now have to take precautionary measures. Popular online messaging and discussion channels such as Discord and Telegram could be exploited by terrorist groups and other bad actors who are increasingly using multiple communication services to create and conduct their propaganda campaigns.
A Discord spokesperson declined to comment. Telegram didn’t respond to a request for comment.
A demonstrator places flowers on white-shrouded body bags representing victims in the Israel-Hamas conflict, in front of the White House in Washington, DC, on November 15, 2023.
Mandel Ngan | AFP | Getty Images
On kids gaming site Roblox, thousands of users recently attended pro-Palestinian protests held within the virtual world. That has required the company to closely monitor for posts that violate its community standards, a Roblox spokesperson told CNBC in a statement.
Roblox has thousands of moderators and “automated detection tools in place to monitor,” the spokesperson said, adding that the site “allows for expressions of solidarity,” but does “not allow for content that endorses or condones violence, promotes terrorism or hatred against individuals or groups, or calls for supporting a specific political party.”
When it comes to looking for talent in the trust and safety space, there’s no shortage. Many of Wagner’s former colleagues at Meta lost their jobs and remain dedicated to the cause.
One of her first investments was in a startup called Cove, which was founded by former Meta trust and safety staffers. Cove is among a handful of emerging companies developing technology that they can sell to organizations, following an established enterprise software model. Other Meta veterans have recently started Cinder and Sero AI to go after the same general market.
“It adds some more coherence to the information ecosystem,” Wagner, who is also a senior advisor at the Responsible Innovation Labs nonprofit, said regarding the new crop of trust and safety tools. “They provide some level of standardized processes across companies where they can access tools and guidelines to be able to manage user-generated content effectively.”
‘Brilliant people out there’
It’s not just ex-Meta staffers who recognize the opportunity.
The founding team of startup TrustLab came from companies including Google, Reddit and TikTok parent ByteDance. And the founders of Intrinsic previously worked on trust and safety-related issues at Apple and Discord.
For the TrustCon conference in July, tech policy wonks and other industry experts headed to San Francisco to discuss the latest hot topics in online trust and safety, including their concerns about the potential societal effects of layoffs across the industry.
Several startups showcased their products in the exhibition hall, promoting their services, talking to potential clients and recruiting talent. ActiveFence, which describes itself as a “leader in providing Trust & Safety solutions to protect online platforms and their users from malicious behavior and content,” had a booth at the conference. So did Checkstep, a content moderation platform.
Cove also had an exhibit at the event.
“I think the cost-cutting has definitely obviously affected the labor markets and the hiring market,” said Cove CEO Michael Dworsky, who co-founded the company in 2021 after more than three years at Facebook. “There are a bunch of brilliant people out there that we can now hire.”
Cove has developed software to help manage a company’s content policy and review process. The management platform works alongside various content moderation systems, or classifiers, to detect issues such as harassment, so businesses can protect their users without needing expensive engineers to develop the code. The company, which counts anonymous social media apps YikYak and Sidechat as customers, says on its website that Cove is “the solution we wish we had at Meta.”
“When Facebook started really investing in trust and safety, it’s not like there were tools on the market that they could have bought,” said Cove technology chief Mason Silber, who previously spent seven years at Facebook. “They didn’t want to build, they didn’t want to become the experts. They did it more out of necessity than desire, and they built some of the most robust, trusted safety solutions in the world.”
A Meta spokesperson declined to comment for this story.
Wagner, who left Meta in mid-2022 after about two and a half years at the company, said that earlier content moderation was more manageable than it is today, particularly with the current Middle East crisis. In the past, for instance, a trust and safety team member could analyze a picture and determine whether it contained false information through a fairly routine scan, she said.
But the quantity and speed of photos and videos being uploaded and the ability of people to manipulate details, especially as generative AI tools become more mainstream, has created a whole new hassle.
Social media sites are now dealing with a swarm of content related to two simultaneous wars, one in the Middle East and another between Russia and Ukraine. On top of that, they have to get ready for the 2024 presidential election in less than a year. Former President Donald Trump, who is under criminal indictment in Georgia for alleged interference in the 2020 election, is the front-runner to become the Republican nominee.
Manu Aggarwal, a partner at research firm Everest Group, said trust and safety is among the fastest-growing segments of a part of the market called business process services, which includes the outsourcing of various IT-related tasks and call centers.
By 2024, Everest Group projects the overall business process services market to be about $300 billion, with trust and safety representing about $11 billion of that figure. Companies such as Accenture and Genpact, which offer outsourced trust and safety services and contract workers, currently capture the bulk of spending, primarily because Big Tech companies have been “building their own” tools, Aggarwal said.
As startups focus on selling packaged and easy-to-use technology to a wider swath of clients, Everest Group practice director Abhijnan Dasgupta estimates that spending on trust and safety tools could be between $750 million and $1 billion by the end of 2024, up from $500 million in 2023. This figure is partly dependent on whether companies adopt more AI services, thus requiring them to potentially abide by emerging AI regulations, he added.
Tech investors are circling the opportunity. Venture capital firm Accel is the lead investor in Cinder, a two-year-old startup whose founders helped build much of Meta’s internal trust and safety systems and also worked on counterterrorism efforts.
“What better team to solve this challenge than the one that played a major role in defining Facebook’s Trust and Safety operations?” Accel’s Sara Ittelson said in a press release announcing the financing in December.
Ittelson told CNBC that she expects the trust and safety technology market to grow as more platforms see the need for greater protection and as the social media market continues to fragment.
The European Commission is now requiring large online platforms with big audiences in the EU to document and detail how they moderate and remove illegal and violent content on their services or face fines of up to 6% of their annual revenue.
Cinder and Cove are promoting their technologies as ways that online businesses can streamline and document their content moderation procedures to comply with the EU’s new regulations, called the Digital Services Act.
‘Frankenstein’s monster’
In the absence of specialized tech tools, Cove’s Dworsky said, many companies have tried to customize Zendesk, which sells customer support software, and Google Sheets to capture their trust and safety policies. That can result in a “very manual, unscalable approach,” he said, describing the process for some companies as “rebuilding and building a Frankenstein’s monster.”
Still, industry experts know that even the most effective trust and safety technologies aren’t a panacea for a problem as big and seemingly uncontrollable as the spread of violent content and disinformation. According to a survey published last week by the Anti-Defamation League, 70% of respondents said that on social media, they’d been exposed to at least one of several types of misinformation or hate related to the Israel-Hamas conflict.
As the problem expands, companies are dealing with the constant struggle over determining what constitutes free speech and what crosses the line into unlawful, or at least unacceptable, content.
Alex Goldenberg, the lead intelligence analyst at the Network Contagion Research Institute, said that in addition to doing their best to maintain integrity on their sites, companies should be honest with their users about their content moderation efforts.
“There’s a balance that is tough to strike, but it is strikable,” he said. “One thing I would recommend is transparency at a time where third-party access and understanding to what is going on at scale on social platforms is what is needed.”
Noam Bardin, the former CEO of navigation firm Waze, now owned by Google, founded the social news-sharing and real-time messaging service Post last year. Bardin, who’s from Israel, said he’s been frustrated with the spread of misinformation and disinformation since the war began in October.
“The whole perception of what’s going on is fashioned and managed through social media, and this means there’s a tremendous influx of propaganda, disinformation, AI-generated content, bringing content from other conflicts into this conflict,” Bardin said.
Bardin said that Meta and X have struggled to manage and remove questionable posts, a challenge that’s become even greater with the influx of videos.
At Post, which is most similar to Twitter, Bardin said he’s been incorporating “all these moderation tools, automated tools and processes” since his company’s inception. He uses services from ActiveFence and OpenWeb, which are both based in Israel.
“Basically, anytime you comment or you post on our platform, it goes through it,” Bardin said regarding the trust and safety software. “It looks at it from an AI perspective to understand what it is and to rank it in terms of harm, pornography, violence, etc.”
Post is an example of the kinds of companies that trust and safety startups are focused on. Active online communities with live-chatting services have also emerged on video game sites, online marketplaces, dating apps and music streaming sites, opening them up to potentially harmful content from users.
Brian Fishman, co-founder of Cinder, said “militant organizations” rely on a network of services to spread propaganda, including platforms like Telegram, and sites such as Rumble and Vimeo, which have less advanced technology than Facebook.
Representatives from Rumble and Vimeo didn’t respond to requests for comment.
Fishman said customers are starting to see trust and safety tools as almost an extension of their cybersecurity budgets. In both cases, companies have to spend money to prevent possible disasters.
“Some of it is you’re paying for insurance, which means that you’re not getting full return on that investment every day,” Fishman said. “You’re investing a little bit more during black times, so that you got capability when you really, really need it, and this is one of those moments where companies really need it.”
The position was valued at about $160 million as of Wednesday’s close.
Tripadvisor shares have been flat since the start of the year after plummeting more than 30% in 2024. Last year, the travel review and booking company said it created a special committee to explore potential options.
Read more CNBC tech news
Starboard Value has gained a reputation for pushing for changes such as new CEOs and cost cuts by acquiring significant shares in companies.
People stand in front of an Apple store in Beijing, China, on April 9, 2025.
Tingshu Wang | Reuters
Apple iPhone sales in China rose in the second quarter of the year for the first time in two years, Counterpoint Research said, as the tech giant looks to turnaround its business in one of its most critical markets.
Sales of iPhones in China jumped 8% year-on-year in the three months to the end of June, according to Counterpoint Research. It’s the first time Apple has recorded growth in China since the second quarter of 2023.
Apple’s performance was boosted by promotions in May as Chinese e-commerce firms discounted Apple’s iPhone 16 models, its latest devices, Counterpoint said. The tech giant also increased trade-in prices for some iPhone.
“Apple’s adjustment of iPhone prices in May was well timed and well received, coming a week ahead of the 618 shopping festival,” Ethan Qi, associate director at Counterpoint said in a press release. The 618 shopping festival happens in China every June and e-commerce retailers offer heavy discounts.
Apple’s return to growth in China will be welcomed by investors who have seen the company’s stock fall around 15% this year as it faces a number of headwinds.
Since then, Huawei has aggressively launched devices in China and has even begun dipping its toe back into international markets. The Chinese tech giant has found success eating away at some of Apple’s market share in China.
Huawei’s sales rose 12% year-on-year in the second-quarter, according to Counterpoint. The firm was the biggest player in China by market share in the second quarter, followed by Vivo and then Apple in third place.
“Huawei is still riding high on core user loyalty as they replace their old phones for new Huawei releases,” Counterpoint Senior Analyst Ivan Lam said.
Chinese tech giant Baidu has bolstered its core search platform with artificial intelligence in the biggest overhaul of the product in 10 years.
Analysts told CNBC the move was a bid to keep ahead of fast-moving rivals like DeepSeek, rather than traditional search players.
“There has been some small pressure on the search business but the focus on AI and Ernie Bot is a key move ahead,” Dan Ives, global head of tech research at Wedbush Securities, told CNBC by email. Ernie Bot is Baidu’s AI chatbot.
“Baidu is not waiting around to watch the paint dry, full steam ahead on AI,” he added.
Baidu AI overhaul
Baidu is China’s biggest search engine, but — as is also being seen by Google — the search market is being disrupted.
Users are flocking instead to AI services such as ChatGPT or DeepSeek, which shocked the world this year with its advanced model it claimed was created at a fraction of the cost of rivals.
But Kai Wang, Asia equity market strategist at Morningstar, also noted that short video platforms such as Douyin and Kuaishou are also getting into AI search and piling pressure on Baidu.
To counter this, Baidu made some major changes to its core search product:
Users can now enter more than a thousand characters in the search box, versus 28 previously;
Questions can be asked in a more direct and conversational manner, mirroring how people now use chatbots;
Users can ask questions through voice but also prompt the seach engine with pictures and files;
Baidu has integrated its AI chatbot features, which enable users to generate photos, text and videos, into the product.
“This is more aligned with how people use ChatGPT and DeepSeek in terms of how they look for answers,” Wang said.
Outside of China, Google has also been looking to enhance its core search product with AI, highlighting how search has been under pressure from the burgeoning technology.
Baidu on the offense
Baidu was one of China’s first movers when it came to AI, releasing its first models and ChatGPT-style product Ernie Bot to the public in 2023. Since then, it has aggressively launched updated AI models.
However, the Beijing-headquartered company has also faced intense competition from fellow tech giants like Alibaba and Tencent, as well as upstarts such as DeepSeek.
These companies have also been launching new models and infusing AI into their products and Baidu’s stock has fallen behind as a result. Baidu shares have risen around 2.5% this year, versus a 30.5% surge for Alibaba and a 20% rise for Tencent.
“This is a defensive and offensive move … Baidu needs to be aggressive and perception-wise show they are not the little brother to Tencent on the AI front,” Wedbush Securities’ Ives added.