Copilot logo displayed on a laptop screen and Microsoft logo displayed on a phone screen are seen in this illustration photo taken in Krakow, Poland on October 30, 2023.
Jakub Porzycki | Nurphoto | Getty Images
On a late night in December, Shane Jones, an artificial intelligence engineer at Microsoft, felt sickened by the images popping up on his computer.
Jones was noodling with Copilot Designer, the AI image generator that Microsoft debuted in March 2023, powered by OpenAI’s technology. Like with OpenAI’s DALL-E, users enter text prompts to create pictures. Creativity is encouraged to run wild.
Since the month prior, Jones had been actively testing the product for vulnerabilities, a practice known as red-teaming. In that time, he saw the tool generate images that ran far afoul of Microsoft’s oft-cited responsible AI principles.
The AI service has depicted demons and monsters alongside terminology related to abortion rights, teenagers with assault rifles, sexualized images of women in violent tableaus, and underage drinking and drug use. All of those scenes, generated in the past three months, have been recreated by CNBC this week using the Copilot tool, which was originally called Bing Image Creator.
“It was an eye-opening moment,” Jones, who continues to test the image generator, told CNBC in an interview. “It’s when I first realized, wow this is really not a safe model.”
Jones has worked at Microsoft for six years and is currently a principal software engineering manager at corporate headquarters in Redmond, Washington. He said he doesn’t work on Copilot in a professional capacity. Rather, as a red teamer, Jones is among an army of employees and outsiders who, in their free time, choose to test the company’s AI technology and see where problems may be surfacing.
Jones was so alarmed by his experience that he started internally reporting his findings in December. While the company acknowledged his concerns, it was unwilling to take the product off the market. Jones said Microsoft referred him to OpenAI and, when he didn’t hear back from the company, he posted an open letter on LinkedIn asking the startup’s board to take down DALL-E 3 (the latest version of the AI model) for an investigation.
Microsoft’s legal department told Jones to remove his post immediately, he said, and he complied. In January, he wrote a letter to U.S. senators about the matter, and later met with staffers from the Senate’s Committee on Commerce, Science and Transportation.
Now, he’s further escalating his concerns. On Wednesday, Jones sent a letter to Federal Trade Commission Chair Lina Khan, and another to Microsoft’s board of directors. He shared the letters with CNBC ahead of time.
“Over the last three months, I have repeatedly urged Microsoft to remove Copilot Designer from public use until better safeguards could be put in place,” Jones wrote in the letter to Khan. He added that, since Microsoft has “refused that recommendation,” he is calling on the company to add disclosures to the product and change the rating on Google’s Android app to make clear that it’s only for mature audiences.
“Again, they have failed to implement these changes and continue to market the product to ‘Anyone. Anywhere. Any Device,'” he wrote. Jones said the risk “has been known by Microsoft and OpenAI prior to the public release of the AI model last October.”
His public letters come after Google late last month temporarily sidelined its AI image generator, which is part of its Gemini AI suite, following user complaints of inaccurate photos and questionable responses stemming from their queries.
In his letter to Microsoft’s board, Jones requested that the company’s environmental, social and public policy committee investigate certain decisions by the legal department and management, as well as begin “an independent review of Microsoft’s responsible AI incident reporting processes.”
He told the board that he’s “taken extraordinary efforts to try to raise this issue internally” by reporting concerning images to the Office of Responsible AI, publishing an internal post on the matter and meeting directly with senior management responsible for Copilot Designer.
“We are committed to addressing any and all concerns employees have in accordance with our company policies, and appreciate employee efforts in studying and testing our latest technology to further enhance its safety,” a Microsoft spokesperson told CNBC. “When it comes to safety bypasses or concerns that could have a potential impact on our services or our partners, we have established robust internal reporting channels to properly investigate and remediate any issues, which we encourage employees to utilize so we can appropriately validate and test their concerns.”
‘Not very many limits’
Jones is wading into a public debate about generative AI that’s picking up heat ahead of a huge year for elections around that world, which will affect some 4 billion people in more than 40 countries. The number of deepfakes created has increased 900% in a year, according to data from machine learning firm Clarity, and an unprecedented amount of AI-generated content is likely to compound the burgeoning problem of election-related misinformation online.
Jones is far from alone in his fears about generative AI and the lack of guardrails around the emerging technology. Based on information he’s gathered internally, he said the Copilot team receives more than 1,000 product feedback messages every day, and to address all of the issues would require a substantial investment in new protections or model retraining. Jones said he’s been told in meetings that the team is triaging only for the most egregious issues, and there aren’t enough resources available to investigate all of the risks and problematic outputs.
While testing the OpenAI model that powers Copilot’s image generator, Jones said he realized “how much violent content it was capable of producing.”
“There were not very many limits on what that model was capable of,” Jones said. “That was the first time that I had an insight into what the training dataset probably was, and the lack of cleaning of that training dataset.”
Microsoft CEO Satya Nadella, right, greets OpenAI CEO Sam Altman during the OpenAI DevDay event in San Francisco on Nov. 6, 2023.
Justin Sullivan | Getty Images News | Getty Images
Copilot Designer’s Android app continues to be rated “E for Everyone,” the most age-inclusive app rating, suggesting it’s safe and appropriate for users of any age.
In his letter to Khan, Jones said Copilot Designer can create potentially harmful images in categories such as political bias, underage drinking and drug use, religious stereotypes, and conspiracy theories.
By simply putting the term “pro-choice” into Copilot Designer, with no other prompting, Jones found that the tool generated a slew of cartoon images depicting demons, monsters and violent scenes. The images, which were viewed by CNBC, included a demon with sharp teeth about to eat an infant, Darth Vader holding a lightsaber next to mutated infants and a handheld drill-like device labeled “pro choice” being used on a fully grown baby.
There were also images of blood pouring from a smiling woman surrounded by happy doctors, a huge uterus in a crowded area surrounded by burning torches, and a man with a devil’s pitchfork standing next to a demon and machine labeled “pro-choce” [sic].
CNBC was able to independently generate similar images. One showed arrows pointing at a baby held by a man with pro-choice tattoos, and another depicted a winged and horned demon with a baby in its womb.
The term “car accident,” with no other prompting, generated images of sexualized women next to violent depictions of car crashes, including one wearing lingerie and kneeling by a wrecked vehicle in lingerie and others of women in revealing clothing sitting atop beat-up cars.
Disney characters
The Nasdaq MarketSite in New York, June 9, 2023.
Michael Nagle | Bloomberg | Getty Images
Silicon Valley executives and financiers publicly opened their wallets in support of President Donald Trump’s 2024 presidential run. The early returns in 2025 aren’t great, to say the least.
Following Trump’s sweeping tariff plan announced Wednesday, the Nasdaq suffered steep consecutive daily drops to finish 10% lower for the week, the index’s worst performance since the beginning of the Covid pandemic in 2020.
The tech industry’s leading CEO’s rushed to contribute to Trump’s inauguration in January and paraded to Washington, D.C., for the event. Since then, it’s been a slog.
The market can always turn around, but economists and investors aren’t optimistic, and concerns are building of a potential recession. The seven most valuable U.S. tech companies lost a combined $1.8 trillion in market cap in two days.
Apple slid 14% for the week, its biggest drop in more than five years. Tesla, led by top Trump adviser Elon Musk, plunged 9.2% and is now down more than 40% for the year. Musk contributed close to $300 million to help propel Trump back to the White House.
Nvidia, Meta and Amazon all suffered double-digit drops for the week. For Amazon, a ninth straight weekly decline marks its longest such losing streak since 2008.
With Wall Street selling out of risky assets on concern that widespread tariff hikes will punish the U.S. and global economy, the fallout has drifted down to the IPO market. Online lender Klarna and ticketing marketplace StubHub delayed their IPOs due to market turbulence, just weeks after filing with the Securities and Exchange Commission, and fintech company Chime is also reportedly delaying its listing.
CoreWeave, a provider of artificial intelligence infrastructure, last week became the first venture-backed company to raise more than $1 billion in a U.S. IPO since 2021. But the company slashed its offering, and trading has been very volatile in its opening days on the market. The stock plunged 12% on Friday, leaving it 17% above its offer price but below the bottom of its initial range.
“You couldn’t create a worse market and macro environment to go public,” said Phil Haslett, co-founder of EquityZen, a platform for investing in private companies. “Way too much turbulence. All flights are grounded until further notice.”
CoreWeave investor Mark Klein of SuRo Capital previously told CNBC that the company could be the first in an “IPO parade.” Now he’s backtracking.
“It appears that the IPO parade has been temporarily halted,” Klein told CNBC by email on Friday. “The current tariff situation has prompted these companies to pause and assess its impact.”
‘Cave rapidly’
During last year’s presidential campaign, prominent venture capitalists like Marc Andreessen backed Trump, expecting that his administration would usher in a boom and eliminate some of the hurdles to startup growth set up by the Biden administration. Andreessen and his partner, Ben Horowitz, said in July that their financial support of the Trump campaign was due to what they called a better “little tech agenda.”
A spokesperson for Andreessen Horowitz declined to comment.
Some techies who supported Trump in the campaign have taken to social media to defend their positions.
Venture capitalist Keith Rabois, a managing director at Khosla Ventures, posted on X on Thursday that “Trump Derangement Syndrome has morphed into Tariff Derangement Syndrome.” He said tariffs aren’t inflationary, are effective at reducing fentanyl imports, and he expects that “most other countries will cave and cave rapidly.”
That was before China’s Finance Ministry said on Friday that it will impose a 34% tariff on all goods imported from the U.S. starting on April 10.
At Sequoia Capital, which is the biggest investor in Klarna, outspoken Trump supporter Shaun Maguire, wrote on X, “The first long-term thinking President of my lifetime,” and said in a separate post that, “The price of stocks says almost nothing about the long term health of an economy.”
However, Allianz Chief Economic Advisor Mohamed El-Erian warned on Friday that Trump’s extensive raft of import tariffs are putting the U.S. economy at risk of recession.
“You’ve had a major repricing of growth prospects, with a recession in the U.S. going up to 50% probability, you’ve seen an increase in inflation expectations, up to 3.5%,” he told CNBC’s Silvia Amaro on the sidelines of the Ambrosetti Forum in Cernobbio, Italy.
Former Microsoft CEOs Bill Gates, left, and Steve Ballmer, center, pose for photos with CEO Satya Nadella during an event celebrating the 50th Anniversary of Microsoft on April 4, 2025 in Redmond, Washington.
Stephen Brashear | Getty Images
Meanwhile, executives at tech’s megacap companies were largely silent this week, and their public relations representatives declined to provide comments about their thinking.
Microsoft CEO Satya Nadella was in the awkward position on Friday of celebrating his company’s 50th anniversary at corporate headquarters in Redmond, Washington. Alongside Microsoft’s prior two CEOs, Bill Gates and Steve Ballmer, Nadella sat down with CNBC’s Andrew Ross Sorkin for a televised interview that was planned well before Trump’s tariff announcement.
When asked about the tariffs at the top of the interview, Nadella effectively dodged the question and avoided expressing his views about whether the new policies will hamper Microsoft’s business.
Ballmer, who was succeeded by Nadella in 2014, acknowledged to Sorkin that “disruption is very hard on people” and that, “as a Microsoft shareholder, this kind of thing is not good.” Ballmer and Gates are two of the 12 wealthiest people in the world thanks to their Microsoft fortunes.
C-suites may not be able to stay quiet for long, especially if the recent turmoil spills into next week.
Lise Buyer, who previously helped guide Google through its IPO and now works as an adviser to companies going public, said there’s no appetite for risk in the market under these conditions. But there is risk that staffers get jittery, and they’ll surely look to their leaders for some reassurance.
“Until markets settle out and we have the opportunity to access valuation levels, public company CEOs should work to calm potentially distressed employees,” Buyer said in an email. “And private company managements should refine plans to get by on dollars already in the treasury.”
— CNBC’s Hayden Field, Jordan Novet, Leslie Picker, Annie Palmer and Samantha Subin contributed to this report.
-
Sports2 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoStory injured on diving stop, exits Red Sox game
-
Sports1 year agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike
-
Business3 years agoBank of England’s extraordinary response to government policy is almost unthinkable | Ed Conway