In the wake of another huge Made by Google event – check out our Pixel 9 Pro hands-on and Pixel Watch 3 hands-on for some of the details – Google has also pushed out some significant AI-powered upgrades for Pixel phones old and new.
There are four upgrades in total, as outlined in a blog post by Google. First is Guided Frame, which gives spoken guidance when users are framing a shot – saying when there’s a face in the frame, and even warning about low light, for example.
Guided Frame has been around for a while to help those with impaired vision, but it’s moving from the Accessibility settings in Android and will now be available from the main camera settings. Previously, you needed to enable the screen reader to get to it.
In addition, Google is promising improvements in object recognition, face filtering in group photos, and object focus. It should mean a more useful and more accurate Guided Frame experience that’s easier to get to.
Next, the Pixel-exclusive Magnifier feature – that essentially lets you use your phone camera like a powerful magnifier to better see the world around you – is getting better, too. A new selfie mode is being added, which helps when touching up make-up, for example.
The Magnifier is also getting the ability to search for certain words in your surroundings (like an airport gate number), a picture-in-picture mode so you can still see the wider context, and the ability to swap between camera lenses.
Captions and transcriptions
Next up we’ve got Live Caption (which captions videos on Android in real-time, using AI) and Live Transcribe (which works in a similar way, but turns spoken audio into text while instantly translating it).
Seven new languages are being added to both these features – Korean, Polish, Portuguese, Russian, Chinese, Turkish, and Vietnamese – together with offline access for Live Transcribe, so you can use them in foreign climes without an internet connection.
If you’ve got a foldable phone – maybe something like the brand new Google Pixel 9 Pro Fold, perhaps – then there’s a new dual-screen mode for Live Transcribe as well. It means one screen can show one language, while around the back of the phone the other display shows the other language.
“With dual-screen mode, you can easily set your phone in a tabletop posture on any surface for better visibility of transcriptions,” says Google. “Now everyone around the table can follow the conversation – whether you’re attending a meeting or having a dinner conversation with friends.”
It’s not clear from Google’s blog posts which older Pixel phones are getting these features, but presumably if they’re already available on your handset at the moment, you’ll be getting the updates when they roll out. For the Magnifier feature specifically, available through the Google Play Store, you’ll need a Pixel 5 or up.
We’re still picking through all the announcements from the Made by Google event, as well as forming our thoughts on the new hardware Google has unveiled. We also saw some pretty major updates for the Google Gemini AI assistant.