IMAGINE an AI-powered future where smartphones are an ancient relic, your 100-inch TV only cost a dollar or pound, and you control your oven simply by thinking about it.
That’s the mind-bending vision of what’s to come, told exclusively to The Sun’s Sean Keach by Meta’s chief technology officer, Andrew Bosworth – watch the full interview above.
19
19
19
Boz, as the future-gazing tech genius is usually known, joined Meta – then Facebook – as a software engineer back in 2006. He even helped create the News Feed.
These days, Boz is a kind of chief lieutenant to Mark Zuckerberg, heading up Meta’s sci-fi-made-real Reality Labs division.
It’s responsible for the company’s virtual- and mixed-reality headsets, as well as the AI-powered Meta Ray-Ban smart glasses.
Now the company is showing off what’s next: Orion, a prototype pair of holographic AI specs controlled by your brain. Yes, really.
Zuckerberg unveiled the 98-gram eyeglasses at his Meta Connect 2024 event on September 25, and I sat down with Boz at Meta HQ in California to find out all about them.
WHAT IS ORION?
Boz told me that the Orion smart glasses were “10 years in the making” – or “five and a half” since the project officially began.
“At the outset, it was a moonshot. It was a north star that we didn’t know if we could hit, but aiming at it, we figured we’d accomplish something of value,” Boz tells me, deep in the heart of Meta’s sprawling Menlo Park campus in Silicon Valley.
“Kind of against all odds – to be honest with you – we’ve hit it.
“We’ve actually built the thing that we wanted to build. And it’s pretty spectacular, the last couple of months, to be able to play with it.”
The Orion glasses are augmented reality specs, which means you see the real world like normal – but it’s overlaid with computer-generated images.
So maybe you pull up a video call in Facebook Messenger or a text chat in WhatsApp, and it’s right there floating in front of you.
And you don’t have to physically touch anything.
It’s all controlled by hand- and eye-tracking, as well as a brain-powered “neural interface”. More on that later.
REPLACING THE SMARTPHONE
The obvious consequence to being able to summon virtually anything, well, virtually is the death of the smartphone.
19
19
It feels impossible to imagine not holding that addictive hunk of metal and glass every single day. But Boz thinks we’ve finally glimpsed what’s next.
“I think this is probably the first device we’ve ever used that you can imagine replacing the phone and allowing you to be – at once – much more connected to the world around you,” Boz explains, proudly.
“Because the device itself is also aware of the world around you, thanks to these tremendous advances in AI.”
He thinks smartphones might still longer on as a kind of retro device with novelty appeal – think the Tamagotchi, or vinyl records.
But the smartphone looks set to fade away…eventually.
“People are still using VHS recorders. I think it’ll be quite a long time before it happens,” Boz jokes.
“To be honest with you, I think these glasses – it’s fun that they’re years away and not decades away for once, which is new.
“But even once they hit the mainstream, getting them down to the point of ubiquity, getting the cost down enough so that it really can be a credible replacement for what have become a very affordable and wonderful device and smartphones, it’s gonna be a long journey.
19
19
“We have to build up the ecosystem of applications and software.
“But I think this is the first device I’ve ever used that I’m like, oh, this could do it.
“Like before this, there was nothing. Before this, the smartphone was like the terminal device that we had experienced.
“And this at least gives us a glimpse of: ‘Oh, I think that if I had this – and if it’s the way I could imagine it being – then I wouldn’t need a phone’.”
A $1 TELEVISION
The good news is that once this future does arrive, it might save you a lot of money.
That’s because not only will these glasses replace your smartphone – but probably your TV too.
After all, if you can simulate a massive screen in the blink of an eye, it would feel pretty silly (not to mention wasteful) to have a hulking, energy-draining slab of glass on your wall.
That might also mean a new design for your house, Boz reckons.
19
“I always think back to houses that were built before the 1950s – and they weren’t built with any place to watch TV,” the Meta CTO tells me.
“Why would they have been? TV wasn’t really a major feature.
“It changed architecture when TVs came about and it become a popular pastime in America – and eventually around the world – to gather together socially at the end of the day with your family, with your friends and watch a show or a program or sports.
“How different will our architecture be? Quite a few of the things that we have today, that we’ve manufactured – we’ve cut a tree down or we’ve burned some plastic and then we’ve shipped it over an ocean on a boat and then we’ve put it on a train and then we’ve put it on a truck and then we’ve put it on a drone and then we’ve put it in your house.
“We’ve used every known human mode of transportation.
“We’ve sent it to space and come back and it’s gotten to your house and you don’t need it.
“Like you don’t need any of that. Why couldn’t your entire TV – mined from rare minerals in the earth – why couldn’t it just be a $1 app?
“It’s all bits. There’s no atoms. That’s all possible once you have penetration as a technology.
19
“I think the glasses are years, not decades away.
“That future is a penetration, which is hard to guess at but it’s very real.”
AR, EVERYWHERE!
It’s obviously exciting on a personal level. I held these glasses, and they’re light enough that I’d be happy to don them for hours.
And once the cost comes down (currently there is no retail mark-up, but Boz calls them “pricey”), other people probably will too.
If two people are wearing the Orion glasses, they can sum up a game of Pong between them
Andrew ‘Boz’ Bosworth
In a distant future, you could imagine lots of people wearing them. Maybe even everyone.
Then it goes from being something useful to you personally – to something that changes the world for everybody.
“This is a technology that when it becomes more ubiquitous, the value increases dramatically. There’s a network effect to it,” Boz points out.
“A good example, just in our demo: if two people are wearing the Orion glasses, they can summon up a game of Pong between them.
“And you’ll be playing Pong. And what’s so funny is you don’t realise how much to the people who aren’t wearing Orion glasses, you just look a little bit ridiculous because you are playing a game in space.
“But then you realise it and you don’t care because it’s really fun and engaging.
“And you and this other person are having this tremendously rich shared experience that happens to be completely virtual.
“But to us, it might as well be tangible.”
19
He also talks about how once more people have these glasses, it won’t be such an isolating experience.
That’s a big problem with current AR and VR gadgets – including the Meta Quest 3S and Apple Vision Pro. Unless you stream it to a screen, no one else nearby can see what you’re looking at.
“Another one is definitely co-presence, where you and I are physically here,” says Boz, basically hinting at teleportation.
“It would be very unusual if I could see a third person and you couldn’t see the person.
“That’s not very helpful. But if we both have glasses and we can both see them and our glasses agree that that person is right here, well, now it becomes a completely fluid conversation for us.
“We have all those demos working and they really are special, but they require real penetration of the technology.”
BRAIN POWER
So in this future where we’re all donning smart specs, are we going to be waving our arms around like crazed Minority Report extras?
Maybe a little. But mostly, no.
19
That’s because the Orion smart glasses are controlled by your brain. Sort of.
I ask Boz if the eventual goal is putting chips in our heads to control these gadgets.
Thankfully the answer is probably not: “The challenge I have with invasive procedurs is that it’s not just ‘oh, I have to do open brain surgery’.
“You do it every year or every couple of years, because the body myelinates, the brain myelinates, and attacks these implants.
“And so they lose effectiveness over time. And like, the consumer readiness for an annual brain surgery is likely to be very low.”
Instead, the Orion ships with a wristband that you wear.
And it’s this wristband that tracks tiny gestures ultimately triggered by your thoughts.
It’s clever and, importantly, means there’s no need to drill into your skull.
19
“What’s amazing about the wrist…is you have tremendous neuroplasticity in your motor cortex. The ability to remap your motor cortex to send signals that it didn’t previously expect to send,” Boz says.
“And so you have actually excess bandwidth and a lot of it runs down – and is monitorable at – your wrists.
“So today the neural interface is very simple. You’re making small gestures and we’re picking that up through EMG sensors.
“But over time, you can get to the point where you have to make no gesture at all.
“You’re still sending an intention as if a motor signal, as if you were gonna move your hand. But the hand just doesn’t move and we’re able to pick up on it.
“And I think the ceiling on that technology could be quite high.”
Meta is still at the very early stages of this technology.
But the ultimate aim is to go far beyond just clicking virtual screens with your mind.
19
Eventually it won’t be just the mouse that’s replaced. Say goodbye to your keyboard. And also all of the buttons on your oven. And on your microwave, for that matter.
“So today it’s a few simple gestures, but we can also do handwriting,” Boz reveals.
“And if you had two, you could do typing with no keyboard.
“And so these become a much higher ceiling. So you asked this question: where does this go?
“If we all had not just AR glasses, but neural interfaces for the devices that I do have in my home – things like an oven, a microwave – they don’t need to have a screen or any buttons on them anymore.
“The display would be provided directly to the glasses, and I would manipulate it directly with my neural interfaces.
“And so everything around you becomes capable of having that kind of interaction modality.”
ALWAYS-ON AI ASSISTANT
Of course the Orion glasses aren’t just meant to show you things – but “see” stuff too.
19
Like Boz’s Ray-Ban glasses of today, Orion will also be powered by Meta AI.
Right now, the virtual assistant can translate languages in real-time, tell you about almost anything you’re seeing, and even remind you to buy more cereal because you’re running low.
Compared to what’s coming in the future, those tricks are going to look like Stone Age technology.
“We have some really amazing demos internally where a pair of glasses that are designed to be sensing glasses are paying attention to what’s going on over the course of the day,” Boz explains.
“And then you can ask them questions about your day and you say: ‘Hey, in the interior design meeting, what was the third colour I picked?’
“And it just knows. And you’re like ‘Oh, there was a poster on the wall at work. What did it say?’
“And it says: ‘Oh yeah, there was a kid’s birthday party happening at this time, at this place.’ And so your day becomes queryable.
“And then you take that and – we haven’t done this yet – but you turn that into an agent that’s working on your behalf. And it’s like: ‘Oh don’t forget, your wife left you a post-it just to pick up a coffee on the way.”
19
It’s this AI-powered experience that Boz thinks is going to kill off the smartphone.
A smartphone can do so much – but it cannot see. Not all the time, anyway.
There’s a labour in having to reach for your phone, haul it out from the depths of your pocket, hold it up, and activate the camera. Ugh.
Boz thinks it’s his Orion glasses “seeing” that is going to change everything.
“I think that is actually the thing that will hit us the hardest first, because that is truly an advantage that these devices have that your phone does not have,” Boz explains excitedly.
“Your phone is with you at all times, but it can’t see what you’re seeing.
“It can’t hear what you’re hearing and it doesn’t have the ability to do much proactively on your behalf, as a consequence.
“And these glasses absolutely combined with Meta AI, they are capable of observing and recording and querying and understanding what’s happening.”
19
You get the impression that Boz has had this grand vision of the future for years.
And in a way, he has. But he also admits that he was totally surprised by how AI developed so quickly.
“We really thought when we started out with the Orion program all those years ago, that it would be holograms first and then at some point later on, AI would come. That was just how we imagined it,” Boz says.
“And we were wrong. AI is coming first. AI is actually gonna hit. And so I’m wearing these Ray-Ban Meta glasses that have AI that can query the world around me.
“I can query the images that they see. They only respond today when I query them.
“But we’re not that far from being able to have…we’ve already invested a tremendous amount in the types of sensors that you would need to be running at very low powers all day.
“And it’s just feeding the AI. It’s not feeding: it’s not a sensor that is providing visible information to anybody in the world. It’s just the AI paying attention and able to answer questions for you or proactively work on your behalf.”
He adds: “So I just think the AI – the sensing part of these glasses, the AI parts of these glasses – is gonna be the bigger impact on society sooner.”
THE FUTURE…SOON
19
So what’s next? When are you going to get your hands on it?
The bad news is that Orion is ultimately just a prototype.
But real consumer-ready versions are being built right now, and we’re talking years – not decades.
“We already have the next two versions of this in prototyping that we’re starting to develop to try to bring to consumers,” Boz reveals.
“Hidden from view. And the next ones are lighter. They’re thinner.
“There are some trade-offs. Some things are similar. The field of view might be a little smaller, but it might be brighter at the same time.”
I ask Boz how important is this product for us – normal people still touching computers with our hands. What does it mean for humanity?
WHO IS ANDREW ‘BOZ’ BOSWORTH?
Here’s Meta’s official bio for Boz…
- Andrew Bosworth — or Boz, as most people know him — is the CTO of Meta, leading the Reality Labs team.
- Andrew graduated from Harvard in 2004 before working as a developer on Microsoft Visio for almost two years.
- He joined Mark Zuckerberg at what was then called Facebook in January of 2006 where he created News Feed and many early anti-abuse systems, some of which are still in production.
- After working briefly on optimizing site speed and reliability, Andrew established and ran the company’s six-week bootcamp program designed to help grow the engineering team and maintain its culture.
- He then led the integration of the Messages and Chat products while improving stability, followed by tenures leading the Groups and Messenger teams.
- At various times, he has been the engineering director overseeing Events, Places, Photos, Videos, Timeline, Privacy, Mobile Monetization and Feed Ads.
- Most recently, he served as the VP of Ads and Business Platform where he led engineering, product, research, analytics, and design.
- In 2017, he created the company’s AR/VR organization, now called Reality Labs.
- Andrew leads Meta’s efforts in AR, VR, AI and consumer hardware across Quest, Ray-Ban Stories and more.
“I don’t know if it’s important for humanity in existential terms. But I do think it comes with some really nice properties,” Boz says.
He adds: “I think we’ve got a tremendous opportunity to continue to advance people’s ability to make the digital world work for them in the way that they go about their physical lives.
“And obviously, at the end of the day, you hope that time is spent with people you care about doing things that you love.
“So we think it’s an important piece of technology.
“We think it’s probably the hardest challenge our industry has taken on probably in my lifetime. And we’re just absolutely thrilled that it works.”
19