Google weighs Gemini AI project to tell people their life story using phone data, photos
Getty Images
A team at Google has proposed using AI technology to create a “bird’s-eye” view of users’ lives using mobile phone data such as photographs and searches.
Dubbed “Project Ellmann,” after biographer and literary critic Richard David Ellmann, the idea would be to use LLMs like Gemini to ingest search results, spot patterns in a user’s photos, create a chatbot, and “answer previously impossible questions,” according to a copy of a presentation viewed by CNBC. Ellmann’s aim, it states, is to be “Your Life Story Teller.”
It’s unclear if the company has plans to produce these capabilities within Google Photos, or any other product. Google Photos has more than one billion users and four trillion photos and videos, according to a company blog post.
Project Ellman is just one of many ways Google is proposing to create or improve its products with AI technology. On Wednesday, Google launched its latest “most capable” and advanced AI model yet, Gemini, which in some cases outperformed OpenAI’s GPT-4. The company is planning to license Gemini to a wide range of customers through Google Cloud for them to use in their own applications. One of Gemini’s standout features is that it’s multimodal, meaning it can process and understand information beyond text, including images, video and audio.
A product manager for Google Photos presented Project Ellman alongside Gemini teams at a recent internal summit, according to documents viewed by CNBC. They wrote that the teams spent the past few months determining that large language models are the ideal tech to make this bird’s-eye approach to one’s life story a reality.
Ellmann could pull in context using biographies, previous moments, and subsequent photos to describe a user’s photos more deeply than “just pixels with labels and metadata,” the presentation states. It proposes to be able to identify a series of moments like university years, Bay Area years, and years as a parent.
“We can’t answer tough questions or tell good stories without a bird’s-eye view of your life,” one description reads alongside a photo of a small boy playing with a dog in the dirt.
“We trawl through your photos, looking at their tags and locations to identify a meaningful moment,” a presentation slide reads. “When we step back and understand your life in its entirety, your overarching story becomes clear.”
The presentation said large language models could infer moments like a user’s child’s birth. “This LLM can use knowledge from higher in the tree to infer that this is Jack’s birth, and that he’s James and Gemma’s first and only child.”
“One of the reasons that an LLM is so powerful for this bird’s-eye approach, is that it’s able to take unstructured context from all different elevations across this tree, and use it to improve how it understands other regions of the tree,” a slide reads, alongside an illustration of a user’s various life “moments” and “chapters.”
Presenters gave another example of determining one user had recently been to a class reunion. “It’s exactly 10 years since he graduated and is full of faces not seen in 10 years so it’s probably a reunion,” the team inferred in its presentation.
The team also demonstrated “Ellmann Chat,” with the description: “Imagine opening ChatGPT but it already knows everything about your life. What would you ask it?”
It displayed a sample chat in which a user asks “Do I have a pet?” To which it answers that yes, the user has a dog which wore a red raincoat, then offered the dog’s name and the names of the two family members it’s most often seen with.
Another example for the chat was a user asking when their siblings last visited. Another asked it to list similar towns to where they live because they are thinking of moving. Ellmann offered answers to both.
Ellmann also presented a summary of the user’s eating habits, other slides showed. “You seem to enjoy Italian food. There are several photos of pasta dishes, as well as a photo of a pizza.” It also said that the user seemed to enjoy new food because one of their photos had a menu with a dish it didn’t recognize.
The technology also determined what products the user was considering purchasing, their interests, work, and travel plans based on the user’s screenshots, the presentation stated. It also suggested it would be able to know their favorite websites and apps, giving examples Google Docs, Reddit and Instagram.
A Google spokesperson told CNBC, “Google Photos has always used AI to help people search their photos and videos, and we’re excited about the potential of LLMs to unlock even more helpful experiences. This is a brainstorming concept a team is at the early stages of exploring. As always, we’ll take the time needed to ensure we do it responsibly, protecting users’ privacy as our top priority.”
Big Tech’s race to create AI-driven ‘Memories’
The proposed Project Ellmann could help Google in the arms race among tech giants to create more personalized life memories.
Google Photos and Apple Photos have for years served “memories” and generated albums based on trends in photos.
In November, Google announced that with the help of AI, Google Photos can now group together similar photos and organize screenshots into easy-to-find albums.
Apple announced in June that its latest software update will include the ability for its photo app to recognize people, dogs, and cats in their photos. It already sorts out faces and allows users to search for them by name.
Apple also announced an upcoming Journal App, which will use on-device AI to create personalized suggestions to prompt users to write passages that describe their memories and experiences based on recent photos, locations, music and workouts.
But Apple, Google and other tech giants are still grappling with the complexities of displaying and identifying images appropriately.
For instance, Apple and Google still avoid labeling gorillas after reports in 2015 found the company mislabeling Black people as gorillas. A New York Times investigation this year found Apple and Google’s Android software, which underpins most of the world’s smartphones, turned off the ability to visually search for primates for fear of labeling a person as an animal.
Companies including Google, Facebook and Apple have over time added controls to minimize unwanted memories, but users have reported they sometimes still surface unwanted memories and require the users to toggle through several settings in order to minimize them.
Intel CEO Lip-Bu Tan makes a speech on stage in Taipei, Taiwan May 19, 2025.
Ann Wang | Reuters
President Donald Trump said Monday that he and members of his cabinet met with Intel CEO Lip-Bu Tan, days after he called on the head of the chipmaker to resign. Intel shares rose 2% in extended trading.
“I met with Mr. Lip-Bu Tan, of Intel, along with Secretary of Commerce, Howard Lutnick, and Secretary of the Treasury, Scott Bessent,” Trump wrote in a post on Truth Social. “The meeting was a very interesting one. His success and rise is an amazing story. Mr. Tan and my Cabinet members are going to spend time together, and bring suggestions to me during the next week. Thank you for your attention to this matter!”
An Intel spokesperson confirmed the meeting.
“Earlier today, Mr. Tan had the honor of meeting with President Trump for a candid and constructive discussion on Intel’s commitment to strengthening U.S. technology and manufacturing leadership,” the spokesperson wrote in an email.
Tan has been an Intel director since 2022, and in March he replaced Pat Gelsinger as CEO. Last week Sen. Tom Cotton, R-Ark., questioned Tan’s ties to China. Cotton brought up a past criminal case involving Cadence Design, where Tan had been CEO, and asked whether Intel required Tan to divest from positions in chipmakers linked to the Chinese Communist Party, the People’s Liberation Army and any other concerning entities in China.
Trump’s latest message marks a stark change in tone from last week. In a Truth Social post on Thursday, the president wrote that Tan “is highly CONFLICTED and must resign, immediately. There is no other solution to this problem.”
Intel said in a comment later that day that the company, directors and Tan are “deeply committed to advancing U.S. national and economic security interests.”
The Trump administration has taken a heavy hand in the business world, particularly in the semiconductor market, as the U.S. battles with China for supremacy in artificial intelligence. Over the weekend, Nvidia agreed to pay the federal government a 15% cut in return for receiving export control licenses that will allow it to once again sell its H20 chip to China and Chinese companies. Nvidia CEO Jensen Huang visited Trump in the White House on Friday.
President Trump on Monday said that he initially asked Nvidia for a 20% cut of the chipmaker’s sales to China, but the number came down to 15% after Huang negotiated with him.
“I said, ‘listen, I want 20% if I’m going to approve this for you, for the country,'” Trump said at a news conference in Washington, D.C.
Tan, 65, took over Intel after the struggling chipmaker had failed to gain significant traction in the AI market, which Nvidia dominates, while it was burning cash to build its foundry business for chip manufacturing.
Tan was born in Malaysia and raised in Singapore before moving to the U.S. and receiving a master’s degree from the Massachusetts Institute of Technology. He said in late July that his first few months as Intel’s CEO had not been easy, with layoffs and cuts to the foundry division.
Intel canceled plans for manufacturing sites in Germany and Poland and would slow down development in Ohio, he told employees.
“Turning the company around will take time and require patience,” Tan said on a conference call with analysts in July. “We have a lot to fix in order to move the company forward.”
Intel shares are up 3% this year as of Monday’s close. The S&P 500 is up 8.4%.
— CNBC’s Fred Imbert contributed to this report.
Technology
StubHub IPO is back on for September after ticketing company delayed plans on tariff concerns
StubHub, the ticketing marketplace that spun out of eBay in 2020, has resumed its plans to go public and is now aiming to hold its IPO next month, CNBC has learned.
The company originally paused its IPO plans in April as the stock market was reeling from President Donald Trump’s “liberation day” tariffs. The decision came after StubHub submitted its prospectus in March indicating it would list on the New York Stock Exchange under the ticker “STUB.”
StubHub now expects to kick off its IPO roadshow after Labor Day, Sept. 1, and make its debut later in the month, according to a source familiar with the matter who asked not to be named because the discussions are confidential.
The company didn’t immediately respond to a request for comment.
StubHub also filed an updated IPO prospectus on Monday. It reported revenue growth in the first quarter of 10% from a year earlier to $397.6 million. Operating income came in at $26.8 million for the period, after the company lost $883,000 in the year-ago period, but its net loss widened to $35.9 million from $29.7 million a year ago.
The IPO market has come to life in recent months after an extended dry spell due to high inflation and rising interest rates. A flurry of startups have made their public debuts, including rocket maker Firefly Aerospace, design software company Figma, crypto firm Circle and AI infrastructure provider CoreWeave. Bullish, the cryptocurrency exchange that counts Peter Thiel as an investor, also filed its IPO prospectus last month.
StubHub has been a longtime player in the ticketing industry since its launch in 2000. It was purchased by eBay for $310 million in 2007, but was reacquired by its co-founder Eric Baker in 2020 for $4 billion through his new company Viagogo.
The company had sought a $16.5 billion valuation before it began the IPO process, CNBC previously reported. StubHub didn’t provide an expected pricing range for its shares in the filing.
As it prepares to go public, StubHub is contending with hefty competition in the online ticketing market. In addition to Ticketmaster, which is owned by Live Nation, StubHub is up against secondary market companies, including Vivid Seats, SeatGeek and TicketNetwork
For the first quarter, StubHub reported gross merchandise sales of $2.08 billion, up 15% from a year prior. That was a slowdown from 47% expansion the previous quarter. StubHub said GMS, or the total value paid by buyers for tickets and fulfillment, builds in each quarter and that initial sales for major concert tours typically occur near the end of the year.
-
Sports3 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoStory injured on diving stop, exits Red Sox game
-
Sports2 years agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Sports2 years agoButton battles heat exhaustion in NASCAR debut
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike