Google weighs Gemini AI project to tell people their life story using phone data, photos
Getty Images
A team at Google has proposed using AI technology to create a “bird’s-eye” view of users’ lives using mobile phone data such as photographs and searches.
Dubbed “Project Ellmann,” after biographer and literary critic Richard David Ellmann, the idea would be to use LLMs like Gemini to ingest search results, spot patterns in a user’s photos, create a chatbot, and “answer previously impossible questions,” according to a copy of a presentation viewed by CNBC. Ellmann’s aim, it states, is to be “Your Life Story Teller.”
It’s unclear if the company has plans to produce these capabilities within Google Photos, or any other product. Google Photos has more than one billion users and four trillion photos and videos, according to a company blog post.
Project Ellman is just one of many ways Google is proposing to create or improve its products with AI technology. On Wednesday, Google launched its latest “most capable” and advanced AI model yet, Gemini, which in some cases outperformed OpenAI’s GPT-4. The company is planning to license Gemini to a wide range of customers through Google Cloud for them to use in their own applications. One of Gemini’s standout features is that it’s multimodal, meaning it can process and understand information beyond text, including images, video and audio.
A product manager for Google Photos presented Project Ellman alongside Gemini teams at a recent internal summit, according to documents viewed by CNBC. They wrote that the teams spent the past few months determining that large language models are the ideal tech to make this bird’s-eye approach to one’s life story a reality.
Ellmann could pull in context using biographies, previous moments, and subsequent photos to describe a user’s photos more deeply than “just pixels with labels and metadata,” the presentation states. It proposes to be able to identify a series of moments like university years, Bay Area years, and years as a parent.
“We can’t answer tough questions or tell good stories without a bird’s-eye view of your life,” one description reads alongside a photo of a small boy playing with a dog in the dirt.
“We trawl through your photos, looking at their tags and locations to identify a meaningful moment,” a presentation slide reads. “When we step back and understand your life in its entirety, your overarching story becomes clear.”
The presentation said large language models could infer moments like a user’s child’s birth. “This LLM can use knowledge from higher in the tree to infer that this is Jack’s birth, and that he’s James and Gemma’s first and only child.”
“One of the reasons that an LLM is so powerful for this bird’s-eye approach, is that it’s able to take unstructured context from all different elevations across this tree, and use it to improve how it understands other regions of the tree,” a slide reads, alongside an illustration of a user’s various life “moments” and “chapters.”
Presenters gave another example of determining one user had recently been to a class reunion. “It’s exactly 10 years since he graduated and is full of faces not seen in 10 years so it’s probably a reunion,” the team inferred in its presentation.
The team also demonstrated “Ellmann Chat,” with the description: “Imagine opening ChatGPT but it already knows everything about your life. What would you ask it?”
It displayed a sample chat in which a user asks “Do I have a pet?” To which it answers that yes, the user has a dog which wore a red raincoat, then offered the dog’s name and the names of the two family members it’s most often seen with.
Another example for the chat was a user asking when their siblings last visited. Another asked it to list similar towns to where they live because they are thinking of moving. Ellmann offered answers to both.
Ellmann also presented a summary of the user’s eating habits, other slides showed. “You seem to enjoy Italian food. There are several photos of pasta dishes, as well as a photo of a pizza.” It also said that the user seemed to enjoy new food because one of their photos had a menu with a dish it didn’t recognize.
The technology also determined what products the user was considering purchasing, their interests, work, and travel plans based on the user’s screenshots, the presentation stated. It also suggested it would be able to know their favorite websites and apps, giving examples Google Docs, Reddit and Instagram.
A Google spokesperson told CNBC, “Google Photos has always used AI to help people search their photos and videos, and we’re excited about the potential of LLMs to unlock even more helpful experiences. This is a brainstorming concept a team is at the early stages of exploring. As always, we’ll take the time needed to ensure we do it responsibly, protecting users’ privacy as our top priority.”
Big Tech’s race to create AI-driven ‘Memories’
The proposed Project Ellmann could help Google in the arms race among tech giants to create more personalized life memories.
Google Photos and Apple Photos have for years served “memories” and generated albums based on trends in photos.
In November, Google announced that with the help of AI, Google Photos can now group together similar photos and organize screenshots into easy-to-find albums.
Apple announced in June that its latest software update will include the ability for its photo app to recognize people, dogs, and cats in their photos. It already sorts out faces and allows users to search for them by name.
Apple also announced an upcoming Journal App, which will use on-device AI to create personalized suggestions to prompt users to write passages that describe their memories and experiences based on recent photos, locations, music and workouts.
But Apple, Google and other tech giants are still grappling with the complexities of displaying and identifying images appropriately.
For instance, Apple and Google still avoid labeling gorillas after reports in 2015 found the company mislabeling Black people as gorillas. A New York Times investigation this year found Apple and Google’s Android software, which underpins most of the world’s smartphones, turned off the ability to visually search for primates for fear of labeling a person as an animal.
Companies including Google, Facebook and Apple have over time added controls to minimize unwanted memories, but users have reported they sometimes still surface unwanted memories and require the users to toggle through several settings in order to minimize them.
An attendee wearing a Super Mario costume uses a Nintendo Switch 2 game console while playing a video game during the Nintendo Switch 2 Experience at the ExCeL London international exhibition and convention centre in London, Britain, April 11, 2025.
Isabel Infantes | Reuters
Nintendo on Friday announced that retail preorder for its Nintendo Switch 2 gaming system will begin on April 24 starting at $449.99.
Preorders for the hotly anticipated console were initially slated for April 9, but Nintendo delayed the date to assess the impact of the far-reaching, aggressive “reciprocal” tariffs that President Donald Trump announced earlier this month.
Most electronics companies, including Nintendo, manufacture their products in Asia. Nintendo’s Switch 1 consoles were made in China and Vietnam, Reuters reported in 2019. Trump has imposed a 145% tariff rate on China and a 10% rate on Vietnam. The latter is down from 46%, after he instituted a 90-day pause to allow for negotiations.
Nintendo said Friday that the Switch 2 will cost $449.99 in the U.S., which is the same price the company first announced on April 2.
“We apologize for the retail pre-order delay, and hope this reduces some of the uncertainty our consumers may be experiencing,” Nintendo said in a statement. “We thank our customers for their patience, and we share their excitement to experience Nintendo Switch 2 starting June 5, 2025.”
The Nintendo Switch 2 and “Mario Kart World“ bundle will cost $499.99, the digital version “Mario Kart World” will cost $79.99 and the digital version of “Donkey Kong Bananza” will cost $69.99, Nintendo said. All of those prices remain unchanged from the company’s initial announcement.
However, accessories for the Nintendo Switch 2 will “experience price adjustments,” the company said, and other future changes in costs are possible for “any Nintendo product.”
It will cost gamers $10 more to by the dock set, $1 more to buy the controller strap and $5 more to buy most other accessories, for instance.
WATCH: Nintendo has ‘a lot of work to do’ to convince casual users to upgrade to Switch 2: Kantan Games
An employee walks past a quilt displaying Etsy Inc. signage at the company’s headquarters in the Brooklyn.
Victor J. Blue/Bloomberg via Getty Images
Etsy is trying to make it easier for shoppers to purchase products from local merchants and avoid the extra cost of imports as President Donald Trump’s sweeping tariffs raise concerns about soaring prices.
In a post to Etsy’s website on Thursday, CEO Josh Silverman said the company is “surfacing new ways for buyers to discover businesses in their countries” via shopping pages and by featuring local sellers on its website and app.
“While we continue to nurture and enable cross-border trade on Etsy, we understand that people are increasingly interested in shopping domestically,” Silverman said.
Etsy operates an online marketplace that connects buyers and sellers with mostly artisanal and handcrafted goods. The site, which had 5.6 million active sellers as of the end of December, competes with e-commerce juggernaut Amazon, as well as newer entrants that have ties to China like Temu, Shein and TikTok Shop.
By highlighting local sellers, Etsy could relieve some shoppers from having to pay higher prices induced by President Trump’s widespread tariffs on trade partners. Trump has imposed tariffs on most foreign countries, with China facing a rate of 145%, and other nations facing 10% rates after he instituted a 90-day pause to allow for negotiations. Trump also signed an executive order that will end the de minimis provision, a loophole for low-value shipments often used by online businesses, on May 2.
Temu and Shein have already announced they plan to raise prices late next week in response to the tariffs. Sellers on Amazon’s third-party marketplace, many of whom source their products from China, have said they’re considering raising prices.
Silverman said Etsy has provided guidance for its sellers to help them “run their businesses with as little disruption as possible” in the wake of tariffs and changes to the de minimis exemption.
Before Trump’s “Liberation Day” tariffs took effect, Silverman said on the company’s fourth-quarter earnings call in late February that he expects Etsy to benefit from the tariffs and de minimis restrictions because it “has much less dependence on products coming in from China.”
“We’re doing whatever work we can do to anticipate and prepare for come what may,” Silverman said at the time. “In general, though, I think Etsy will be more resilient than many of our competitors in these situations.”
Still, American shoppers may face higher prices on Etsy as U.S. businesses that source their products or components from China pass some of those costs on to consumers.
Etsy shares are down 17% this year, slightly more than the Nasdaq.
WATCH: Amazon CEO Andy Jassy says sellers will pass cost of tariffs on to consumers
-
Sports2 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoStory injured on diving stop, exits Red Sox game
-
Sports1 year agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment2 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike
-
Business3 years agoBank of England’s extraordinary response to government policy is almost unthinkable | Ed Conway