Hands-on with Meta’s Orion AR glasses prototype and the possible future of computing
Meta CEO Mark Zuckerberg presents Orion AR Glasses, as he makes a keynote speech during the Meta Connect annual event, at the company’s headquarters in Menlo Park, California, U.S. September 25, 2024.
Manuel Orbegozo | Reuters
The most impressive aspect of Meta’s Orion augmented-reality glasses has more to do with size and comfort than flashy computer graphics.
CNBC senior media and tech correspondent Julia Boorstin was able to use Orion this week at Meta’s annual Connect conference, and she was captivated by the prototype’s compact form relative to the various Meta Quest and Apple Vision Pro virtual reality headsets.
“What was really striking to me about these was that they were incredibly lightweight,” Boorstin said.
Meta CEO Mark Zuckerberg revealed the Orion glasses on Wednesday and pitched them as “a glimpse of a future that I think is going to be pretty exciting.” The glasses are black and thick framed and come with a wireless “puck” that allows the device to run apps like a holographic game of digital chess or ping-pong that appear as digital graphics spliced into the real world.
The experimental glasses are part of Zuckerberg’s multi-billion dollar plans to build the next-generation of personal computing for the so-called metaverse, a term used by Meta to describe people interacting with one another online in virtual 3D spaces.
While Orion is not capable of putting users in fully virtual worlds, the glasses can overlay digital graphics onto the real world. And unlike VR headsets that can be cumbersome to wear for extended periods, Boorstin said she found the Orion glasses to be a good fit.
“The form factor didn’t feel meaningfully different than wearing a pair of heavy, ordinary glasses, and they were not uncomfortable to wear,” she said.
Though the current incarnation of the Orion AR glasses could pass as a movie prop for the film “Revenge of the Nerds,” Boorstin said she believes they’re only going to get smaller as technology improves.
“This is the first generation — four years from now, how much smaller will they be?” Boorstin said.
CNBC’s Julia Boorstin tries out Meta’s new Orion AR glasses on Sept. 25th, 2024.
Stephen Desaulniers | CNBC
When wearing the AR glasses, Boorstin was able to see digital holograms displaying the visual icons of apps like Instagram, Facebook and some extras like a browser and a video game mixed with the surroundings inside a small office at Meta’s headquarters.
Boorstin saw those digital icons overlaid atop her real-world surroundings with her own eyes. That’s an improvement over “passthrough” techniques used by current VR devices. For passthrough, companies use cameras on the outside of their headsets to show users a digital representation of the real world blended with computer graphics through their device screens.
Orion is able to overlay digital imagery on the real world using a much more expensive method. Its lenses aren’t made from traditional glass or plastic but rather a refractive material called silicon-carbide. When the Orion’s miniaturized projectors, built-in to the arms of the glasses, beam light into the silicon-carbide lenses, users can see “holograms” in their field of vision, an experience Boorstin said “felt totally normal and very natural.”
When the holograms were turned off, “it felt as if you were wearing glasses or sunglasses, and it wasn’t distracting or nauseating,” Boorstin said.
Boorstin was able to open, close and scroll through the apps with the help of a wristband, that she said felt similar to an old, lightweight Fitbit.
“The wristband can sense your finger and hand movements, so your hand can be by your side,” Boorstin said, describing how her finger movements and gestures manipulated the digital icons. “I was surprised that it was so accurate and that I could figure out these hand motions, and it picked them up exactly.”
In one demo, the Orion glasses were able to identify various food ingredients, like chia seeds, that were spread out on a table. It then projected a suitable recipe that appeared digital above the real-world seeds. In another demo, Boorstin played a simple game of pong, except the video game graphics were projected onto a real-world desk in front of her.
One demo that really impressed her involved seeing her producer’s face digitally appear in front of her while he called from another room. The overall experience of the 3D video call “felt very clear” to Boorstin, who noticed that the graphic’s resolution would change depending on where she placed it within her field of vision. It was enough to startle her into questioning whether or not the producer could actually see her in real life since it appeared as if he was there in front of her (he could not).
“I could see him perfectly, and he could not see me,” Boorstin said. “But I could hear him, and it was like I was FaceTiming with him, but he was in my glasses.”
By experiencing Orion, Boorstin said she has a better sense of how Meta’s research and development is directly benefiting the company’s other products, like its Quest headsets and Ray-Ban smart glasses.
“They’ve been working so hard to make these components teeny, tiny, efficient, weightless,” she said.
Technology
Meta set to unveil first consumer-ready smart glasses with a display, wristband next month
Meta CEO Mark Zuckerberg makes a keynote speech at the Meta Connect annual event at the company’s headquarters in Menlo Park, Calif., on Sept. 25, 2024.
Manuel Orbegozo | Reuters
Meta is planning to use its annual Connect conference next month to announce a deeper push into smart glasses, including the launch of the company’s first consumer-ready glasses with a display, CNBC has learned.
That’s one of the two new devices Meta is planning to unveil at the event, according to people familiar with the matter. The company will also launch its first wristband that will allow users to control the glasses with hand gestures, the people said.
Connect is a two-day conference for developers focused on virtual reality, AR and the metaverse. It was originally called Oculus Connect and obtained its current moniker after Facebook changed its parent company name to Meta in 2021.
The glasses are internally codenamed Hypernova and will include a small digital display in the right lens of the device, said the people, who asked not to be named because the details are confidential.
The device is expected to cost about $800 and will be sold in partnership with EssilorLuxottica, the people said. CNBC reported in October that Meta was working with Luxottica on consumer glasses with a display.
Meta declined to comment. Luxottica, which is based in France and Italy, didn’t respond to a request for comment.
Meta began selling smart glasses with Luxottica in 2021 when the two companies released the first-generation Ray-Ban Stories, which allowed users to take photos or videos using simple voice commands. The partnership has since expanded, and last year included the addition of advanced AI features that made the second generation of the product an unexpected hit with early adopters.
Luxottica owns a number of glasses brands, including Ray-Ban, and licenses many others like Prada. It’s unclear what brand Luxottica will use for the glasses with AR, but a Meta job listing posted this week said the company is looking for a technical program manager for its “Wearables organization,” which “is responsible for the Ray-Ban AR glasses and other wearable hardware.”
In June, CNBC reported that Meta and Luxottica plan to release Prada-branded smart glasses. Prada glasses are known for having thick frames and arms, which could make them a suitable option for the Hypernova device, one of the people said.
Last year, Meta CEO Mark Zuckerberg used Connect to showcase the company’s experimental Orion AR glasses.
The Orion features AR capabilities on both lenses, capable of blending 3D digital visuals into the physical world, but the device served only as a prototype to show the public what could be possible with AR glasses. Still, Orion built some positive momentum for Meta, which since late 2020 has endured nearly $70 billion in losses from its Reality Labs unit that’s in charge of building hardware devices.
With Hypernova, Meta will finally be offering glasses with a display to consumers, but the company is setting low expectations for sales, some of the sources said. That’s because the device requires more components than its voice-only predecessors, and will be slightly heavier and thicker, the people said.
Meta and Ray-Ban have sold 2 million pairs of their second-generation glasses since 2023, Luxottica CEO Francesco Milleri said in February. In July, Luxottica said that revenue from sales of the smart glasses had more than tripled year over year.
As part of an extension agreement between Meta and Luxottica announced in September, Meta obtained a stake of about 3% in the glasses company according to Bloomberg. Meta also gets exclusive rights to Luxottica’s brands for its smart glasses technology for a number of years, a person familiar with the matter told CNBC in June.
Although Hypernova will feature a display, those visual features are expected to be limited, people familiar with the matter said. They said the color display will offer about a 20 degree field of view — meaning it will appear in a small window in a fixed position — and will be used primarily to relay simple bits of information, such as incoming text messages.
Andrew Bosworth, Meta’s technology chief, said earlier this month that there are advantages to having just one display rather than two, including a lower price.
“Monocular displays have a lot going for them,” Bosworth said in an Instagram video. “They’re affordable, they’re lighter, and you don’t have disparity correction, so they’re structurally quite a bit easier.”
‘Interact with an AI assistant’
Other details of Meta’s forthcoming glasses were disclosed in a July letter from the U.S. Customs and Border Patrol to a lawyer representing Meta. While the letter redacted the name of the company and the product, a person with knowledge of the matter confirmed that it was in reference to Meta’s Hypernova glasses.
“This model will enable the user to take and share photos and videos, make phone calls and video calls, send and receive messages, listen to audio playback and interact with an AI assistant in different forms and methods, including voice, display, and manual interactions,” according to the letter, dated July 23.
The letter from CBP was part of routine communication between companies and the U.S. government when determining the country of origin for a consumer product. It refers to the product as “New Smart Glasses,” and says the device will feature “a lens display function that allows the user to interface with visual content arising from the Smart Features, and components providing image data retrieval, processing, and rendering capabilities.”
CBP didn’t provide a comment for this story.
The Hypernova glasses will also come paired with a wristband that will use technology built by Meta’s CTRL Labs, said people familiar with the matter. CTRL Labs, which Meta acquired in 2019, specializes in building neural technology that could allow users to control computing devices using gestures in their arms.
The wristband is expected to be a key input component for the company’s future release of full AR glasses, so getting data now with Hypernova could improve future versions of the wristband, the people said. Instead of using camera sensors to track body movements, as with Apple’s Vision Pro headset, Meta’s wristband uses so-called sEMG sensor technology, which reads and interprets the electrical signals from hand movements.
One of the challenges Meta has faced with the wristband involves how people choose to wear it, a person familiar with the product’s development said. If the device is too loose, it won’t be able to read the user’s electrical signals as intended, which could impact its performance, the person said. Also, the wristband has run into issues in testing related to which arm it’s worn on, how it works on men versus women and how it functions on people who wear long sleeves.
The CTRL Labs team published a paper in Nature in July about its wristband, and Meta wrote about it in a blog post. In the paper, the Meta team detailed its use of machine learning technology to make the wristband work with as many people as possible. The additional data collected by the upcoming device should improve those capabilities for future Meta smart glasses.
“We successfully prototyped an sEMG wristband with Orion, our first pair of true augmented reality (AR) glasses, but that was just the beginning,” Meta wrote in the post. “Our teams have developed advanced machine learning models that are able to transform neural signals controlling muscles at the wrist into commands that drive people’s interactions with the glasses, eliminating the need for traditional—and more cumbersome—forms of input.”
Bloomberg reported the wristband component in January.
Meta has recently started reaching out to developers to begin testing both Hypernova and the accompanying wristband, people familiar with the matter said. The company wants to court third-party developers, particularly those who specialize in generative AI, to build experimental apps that Meta can showcase to drum up excitement for the smart glasses, the people said.
In addition to Hypernova and the wristband, Meta will also announce a third-generation of its voice-only smart glasses with Luxottica at Connect, one person said.
That device was also referenced by CBP in its July letter, referring to it as “The Next Generation Smart Glasses.” The glasses will include “components that provide capacitive touch functionality, allowing users to interact with the Smart Glasses through touch gestures,” the letter said.
WATCH: Elon Musk asked Zuckerberg to join xAI bit for OpenAI
Google CEO Sundar Pichai gestures to the crowd during Google’s annual I/O developers conference in Mountain View, California on May 20, 2025.
Camille Cohen | Afp | Getty Images
Alphabet shares rose on a Friday report that Apple is in early discussions to use Google’s Gemini AI models for an updated version of the iPhone-maker’s Siri assistant.
The company’s shares rose more than 3% on the Bloomberg report, which said Apple recently inquired of Google about the potential for the search giant to build a custom AI model that would power a new Siri that could launch next year. Google’s flagship AI models Gemini have consistently been atop key benchmarks for artificial intelligence advancements while Apple has struggled to define its own AI strategy.
The reported talks come as Google faces potential risk to its lucrative search deals with Apple. This month, a U.S. judge is expected to rule on the penalties for Google’s alleged search monopoly, in which the Department of Justice recommending eliminating exclusionary agreements with third parties. For Google, that refers to its search position on Apple’s iPhone and Samsung devices — deals that cost the company billions of dollars a year in payouts.
The Android maker has said its Gemini models will become the default assistant on Android phones. Google this year has showed Gemini doing capabilities that go beyond Siri’s capabilities, such as summarizing videos.
Craig Federighi, who oversees Apple’s operating systems, said at last year’s developer conference that the iPhone maker would like to add other AI models for specific purposes into its Apple Intelligence framework. Federighi specifically mentioned Google, whose Gemini can now hold conversations with users and handle input that comes from photos, videos, voice or text. Apple is also exploring partnerships with Anthropic and OpenAI as it tried to renew its AI roadmap, according to a June Bloomberg report.
Documents revealed during Google’s remedy trial showed executives from Apple were involved in the negotiations over using Google’s Gemini for a potential search option.
Google declined to comment.
WATCH: Apple explores using Google Gemini AI to power revamped Siri, reports say
-
Sports3 years ago‘Storybook stuff’: Inside the night Bryce Harper sent the Phillies to the World Series
-
Sports1 year agoStory injured on diving stop, exits Red Sox game
-
Sports2 years agoGame 1 of WS least-watched in recorded history
-
Sports2 years agoMLB Rank 2023: Ranking baseball’s top 100 players
-
Sports4 years ago
Team Europe easily wins 4th straight Laver Cup
-
Sports2 years agoButton battles heat exhaustion in NASCAR debut
-
Environment2 years agoJapan and South Korea have a lot at stake in a free and open South China Sea
-
Environment3 years agoGame-changing Lectric XPedition launched as affordable electric cargo bike