Google weighs Gemini AI project to tell people their life story using phone data, photos
Getty Images
A team at Google has proposed using AI technology to create a “bird’s-eye” view of users’ lives using mobile phone data such as photographs and searches.
Dubbed “Project Ellmann,” after biographer and literary critic Richard David Ellmann, the idea would be to use LLMs like Gemini to ingest search results, spot patterns in a user’s photos, create a chatbot, and “answer previously impossible questions,” according to a copy of a presentation viewed by CNBC. Ellmann’s aim, it states, is to be “Your Life Story Teller.”
It’s unclear if the company has plans to produce these capabilities within Google Photos, or any other product. Google Photos has more than one billion users and four trillion photos and videos, according to a company blog post.
Project Ellman is just one of many ways Google is proposing to create or improve its products with AI technology. On Wednesday, Google launched its latest “most capable” and advanced AI model yet, Gemini, which in some cases outperformed OpenAI’s GPT-4. The company is planning to license Gemini to a wide range of customers through Google Cloud for them to use in their own applications. One of Gemini’s standout features is that it’s multimodal, meaning it can process and understand information beyond text, including images, video and audio.
A product manager for Google Photos presented Project Ellman alongside Gemini teams at a recent internal summit, according to documents viewed by CNBC. They wrote that the teams spent the past few months determining that large language models are the ideal tech to make this bird’s-eye approach to one’s life story a reality.
Ellmann could pull in context using biographies, previous moments, and subsequent photos to describe a user’s photos more deeply than “just pixels with labels and metadata,” the presentation states. It proposes to be able to identify a series of moments like university years, Bay Area years, and years as a parent.
“We can’t answer tough questions or tell good stories without a bird’s-eye view of your life,” one description reads alongside a photo of a small boy playing with a dog in the dirt.
“We trawl through your photos, looking at their tags and locations to identify a meaningful moment,” a presentation slide reads. “When we step back and understand your life in its entirety, your overarching story becomes clear.”
The presentation said large language models could infer moments like a user’s child’s birth. “This LLM can use knowledge from higher in the tree to infer that this is Jack’s birth, and that he’s James and Gemma’s first and only child.”
“One of the reasons that an LLM is so powerful for this bird’s-eye approach, is that it’s able to take unstructured context from all different elevations across this tree, and use it to improve how it understands other regions of the tree,” a slide reads, alongside an illustration of a user’s various life “moments” and “chapters.”
Presenters gave another example of determining one user had recently been to a class reunion. “It’s exactly 10 years since he graduated and is full of faces not seen in 10 years so it’s probably a reunion,” the team inferred in its presentation.
The team also demonstrated “Ellmann Chat,” with the description: “Imagine opening ChatGPT but it already knows everything about your life. What would you ask it?”
It displayed a sample chat in which a user asks “Do I have a pet?” To which it answers that yes, the user has a dog which wore a red raincoat, then offered the dog’s name and the names of the two family members it’s most often seen with.
Another example for the chat was a user asking when their siblings last visited. Another asked it to list similar towns to where they live because they are thinking of moving. Ellmann offered answers to both.
Ellmann also presented a summary of the user’s eating habits, other slides showed. “You seem to enjoy Italian food. There are several photos of pasta dishes, as well as a photo of a pizza.” It also said that the user seemed to enjoy new food because one of their photos had a menu with a dish it didn’t recognize.
The technology also determined what products the user was considering purchasing, their interests, work, and travel plans based on the user’s screenshots, the presentation stated. It also suggested it would be able to know their favorite websites and apps, giving examples Google Docs, Reddit and Instagram.
A Google spokesperson told CNBC, “Google Photos has always used AI to help people search their photos and videos, and we’re excited about the potential of LLMs to unlock even more helpful experiences. This is a brainstorming concept a team is at the early stages of exploring. As always, we’ll take the time needed to ensure we do it responsibly, protecting users’ privacy as our top priority.”
Big Tech’s race to create AI-driven ‘Memories’
The proposed Project Ellmann could help Google in the arms race among tech giants to create more personalized life memories.
Google Photos and Apple Photos have for years served “memories” and generated albums based on trends in photos.
In November, Google announced that with the help of AI, Google Photos can now group together similar photos and organize screenshots into easy-to-find albums.
Apple announced in June that its latest software update will include the ability for its photo app to recognize people, dogs, and cats in their photos. It already sorts out faces and allows users to search for them by name.
Apple also announced an upcoming Journal App, which will use on-device AI to create personalized suggestions to prompt users to write passages that describe their memories and experiences based on recent photos, locations, music and workouts.
But Apple, Google and other tech giants are still grappling with the complexities of displaying and identifying images appropriately.
For instance, Apple and Google still avoid labeling gorillas after reports in 2015 found the company mislabeling Black people as gorillas. A New York Times investigation this year found Apple and Google’s Android software, which underpins most of the world’s smartphones, turned off the ability to visually search for primates for fear of labeling a person as an animal.
Companies including Google, Facebook and Apple have over time added controls to minimize unwanted memories, but users have reported they sometimes still surface unwanted memories and require the users to toggle through several settings in order to minimize them.
Technology
Microsoft to invest $1.7 billion into AI infrastructure in Indonesia, CEO Satya Nadella says
Vehicles set to be shipped to Europe, at Taicang Port on Dec. 19, 2022, in Suzhou, China.
Vcg | Visual China Group | Getty Images
The European Union will need to levy higher-than-expected tariffs of up to 55% on Chinese electric vehicles to curb their imports into the bloc, according to a new analysis by Rhodium Group.
The findings, released Monday, come amid the EU’s ongoing anti-subsidy investigation into EV imports from China.
Rhodium Group, which expects the EU to impose tariffs in the 15% to 30% range on Chinese EVs, said those tariffs were unlikely to be enough to check competition from China.
“Even if the duties come in at the higher end of this range, some China-based producers will still be able to generate comfortable profit margins on the cars they export to Europe because of the substantial cost advantages they enjoy,” the report said.
Chinese companies such as BYD, which toppled Tesla to become the world’s largest EV manufacturer last year, can sell cars at much higher rates and profit margins in regions such as the EU compared with the domestic market, despite paying a 10% tariff rate. Chinese EV makers are locked in an intense price war in their home market.
BYD’s Seal U model, which sells for 20,500 euros in China and 42,000 euros in the EU, generates an estimated profit of 1,300 euros in its home market versus 14,300 euros per car in Europe, Rhodium said. Even after 30% in tariffs, a company like BYD will make a higher profit in the EU, it added.
The report said that BYD will likely need to cut prices to meet its goals of gaining more market share in the EU. A 30% tariff rate would still leave enough room to do so.
“Much steeper duties of around 45%, or even 55% for fiercely competitive producers like BYD, would probably be necessary in order to render exports to the European market unappealing on commercial grounds,” the report said.
The EU investigation
The European Commission, the executive arm of the EU, launched a probe into Chinese EVs and subsidies last year, with officials saying that a flood of cheap vehicles threatened domestic producers.
According to some experts, incentives put in place in China in the early 2010s led to a surge in startups and increased battery cell capacity in the country, paving the way for globally competitive and affordable EVs.
Chinese EV makers have already been facing resistance from the U.S. amid high tariffs and political opposition, making the European market more important to companies such as BYD that are pursuing global expansion.
EVs from Chinese companies are expected to make up 11% of the EU’s market in 2024 and could reach 20% by 2027, according to an analysis by the European Federation for Transport and Environment.
When accounting for made-in-China vehicles from non-Chinese-companies, the figure is expected to surpass 25% this year.
Imports of EVs from non-Chinese firms could also come under in the EU subsidy investigation, with Rhodium estimating that duties at the 15%-30% level could wipe out the business for foreign players such BMW or Tesla that ship cars from China.
In response to the policy risks, EV makers have been working on shifting manufacturing to Europe. BYD plans to build a factory in Hungary.
However, Rhodium adds that Brussels could use other means to protect the Europe’s EV industry, such as restricting Chinese imports on national security grounds or increasing consumer subsidies for EU-made vehicles.
The Chinese government has slammed the EU subsidy investigation as “blatant protectionism,” arguing that its companies are simply more competitive than their Western counterparts.
Samsung Electronics Co. Galaxy S24 smartphones during a media preview event in Seoul, South Korea, on Monday, Jan. 15, 2024. Samsung, the world’s most prolific smartphone maker, is leaning into artificial intelligence as the key to unlocking greater sales this year. Photographer: SeongJoon Cho/Bloomberg via Getty Images
SeongJoon Cho | Bloomberg | Getty Images
Samsung Electronics on Tuesday said operating profit for the first quarter jumped 932.8% as memory chip prices rebounded on the back of AI optimism.
Here are Samsung’s first-quarter results versus LSEG estimates:
- Revenue: 71.92 trillion Korean won (about $52.3 billion), vs. 71.04 trillion Korean won
- Operating profit: 6.61 trillion Korean won, vs. 5.94 trillion Korean won
Samsung’s revenue for the quarter ending March jumped 12.81% from a year ago, while operating profit soared 932.8% in the same period.
The figures were in line with the company’s guidance earlier this month, where Samsung said operating profit in the January-March quarter likely rose to 6.6 trillion Korean won, up 931% from a year ago. The firm expected first quarter revenue at 71 trillion won.
The South Korean electronics giant saw record losses in 2023 as the industry reeled from a post-Covid slump in demand.
“The company posted KRW 71.92 trillion in consolidated revenue on the back of strong sales of flagship Galaxy S24 smartphones and higher prices for memory semiconductors. Operating profit increased to KRW 6.61 trillion as the Memory Business returned to profit by addressing demand for high value-added products,” Samsung Electronics said in a statement on Tuesday.
Samsung is the world’s largest manufacturer of dynamic random-access memory chips (DRAM), which are commonly found in a wide range of consumer devices including smartphones and computers.
“We assume the earnings surprise was driven by higher memory price hike on AI-driven strong upturn cycle. We anticipate the company will guide for positive memory market outlook and emphasize its readiness in AI era including HBM (12H HBM3E, HBM4) and foundry/packaging solution,” said SK Kim of Daiwa Capital Markets in emailed comments to CNBC on Monday, ahead of the earnings release.
As AI models become more complex and datasets become larger, these models need memory chips with higher capacities and faster speeds to cater to these workloads.
Kim said in an April 5 report he expects another price hike on memory chips to drive Samsung’s second-quarter earnings on the back of an AI boom and the earthquake in Taiwan.
“Especially, we expect more upside in prices resulting from the earthquake in Taiwan,” said Kim, adding that the earthquake in early April temporarily impacted TSMC‘s and Micron‘s production.
Citi analysts said they see upside for Samsung’s NAND flash memory business as a result of AI computing demand. In a note on April 5, they reiterated their “buy” rating on the firm with a target price of 120,000 won — a 56% upside from the closing price of 76,700 won on Monday.
NAND is another staple memory chip alongside DRAM.
“We expect storage (HDD) to be the next bottleneck in AI computing, especially in AI training, and foresee Samsung Electronics to be one of the key beneficiaries of SSD demand momentum for AI training,” said the Citi analysts.
Growing competition
Many countries in the world are racing to manufacture advanced semiconductors.
Earlier this month, the Biden administration agreed to grant Samsung up to $6.4 billion of funding to create new manufacturing capacity to produce chips in Texas. Micron and TSMC are also poised to receive grants to boost chipmaking in the U.S. after decades of chip production moving to Asia.
Samsung and TSMC are set to face competition from Japan’s Rapidus Corporation, which was recently granted $3.89 billion in additional subsidies from the Japanese government to mass produce 2-nanometer chips from 2027.
There are rising concerns that Samsung Electronics risks losing its leading position to rivals like SK Hynix, the world’s no. 2 memory chip maker.
SK Hynix on March 19 said it became the first in the industry to mass produce HBM3E, the next-generation of high bandwidth memory chips used in AI chipsets. SK Hynix is the primary supplier of HBM3 chips to Nvidia’s AI chipsets.
Mehdi Hosseini, senior tech hardware analyst of Susquehanna International Group, pointed out in early April that Samsung used to be the market leader in memory, smartphones and display innovations.
Now, Samsung is only “benefiting from the cycle recovery,” he added.
In the first quarter, Samsung managed to regain the top spot in smartphone shipments after losing the crown to Apple in 2023, according to International Data Corp.
-
Sports2 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Environment11 months agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Sports3 years ago
Team Europe easily wins 4th straight Laver Cup
-
Sports6 months agoGame 1 of WS least-watched in recorded history
-
Environment1 year agoGame-changing Lectric XPedition launched as affordable electric cargo bike
-
Technology3 years ago
Game consoles were once banned in China. Now Chinese developers want a slice of the $49 billion pie
-
Politics2 years ago
Have the last few wobbly weeks seen a turning point for Johnson as PM?