All the AI features coming to Google’s Pixel 9 series

Google’s Pixel 9 lineup is officially here, and the company can’t stop talking about the AI features it’s got coming.

On the hardware front, Google boosted the RAM for all of its new phones to accommodate memory-hungry on-device AI. The Pixel 9 received 12GB of RAM, while the rest will have 16GB of memory.

But the bigger side of the story is software — the phones all get a handful of new Pixel-exclusive AI features. And the Pro phones are also getting a year of Google One AI Premium, which comes with access to Gemini Advanced, Google’s most capable AI system.

Here’s a rundown of the biggest Pixel 9 AI features Google announced today.

Google’s new Pixel Screenshots feature is a little like Microsoft’s Recall feature that uses AI to track everything you do on your computer via constant screenshots, except it’s more of a manual affair. If you want to keep track of something like an event you’re planning with friends or a recipe to make for dinner, you can take a screenshot and then conversationally search for the information later. Google says this is a Pixel device exclusive.

Google took a cue from Apple, too, giving Gemini the ability to respond to you based on what’s on your phone screen when you ask it. The company says that after summoning Gemini, you can tap “Ask about this screen” or “Ask about this video” to direct the model to what you’re looking at for contextual replies. Google says this means Gemini can do things like add a list of restaurants from a YouTube travel video to Google Maps.

Gemini will be faster as a voice assistant, Google says, thanks to new models such as Gemini 1.5 Flash. The company says this model also offers higher-quality responses and won’t get things wrong quite as often as previous versions of the assistant. Google is also adding more extensions to Gemini, so it will be able to pull information from — or do things in — apps like Keep, Tasks, Utilities, and Google Calendar.

Google is rolling out its own version of ChatGPT’s voice chat, called Gemini Live. The company says the feature enables natural conversation, so you can interrupt Gemini mid-sentence or pause a conversation and pick it up later. Gemini Live will work even if your screen is locked, Google says. It’s only available for Gemini Advanced subscribers, though.

Folks who own the new Pixel Buds Pro 2 can talk to Gemini through their headphones as well by tapping and holding on the headphones and saying, “Let’s talk live.”

Google is adding a new Magic Editor option to “reimagine” a photo by typing out what you want to see and then letting the app transform parts of the image. You can do this with prompts alone or by selecting parts of the image — either by tracing your selection or tapping a region like the sky — and typing what you want to change or add. So, if you wanted to, you could totally replace the sky or other aspects of a background by describing something else entirely. Who needs a perfect day when you can make one, I guess?

Along the same “what is a photo?” lines as the new Magic Editor feature, the Pixel 9 camera’s “Add Me” option lets you easily take group photos without bothering strangers to do it for you. Just take a picture of your friends, then hand your phone off and step into the same spot they were in (using an AR overlay to guide the camera’s framing), and the camera will stitch the two images together afterward. The feature uses an AR overlay of the original picture to guide the framing of the second one.

Google says Add Me will be available on all Pixel 9 phones at launch.

A new Pixel Studio app lets you create illustrations using text prompts. The feature is partially powered by “an on-device diffusion model” and Google’s Imagen 3 cloud-based model. The feature requires an internet connection, according to Google.

Google is adding what it says is an AI-powered ability to share parts of an image or of your screen when you use Circle to Search. After you’ve circled the portion you want, you can tap a new share button to send it via text — it’s basically cropping part of an image to share but without all the usual rigmarole of editing an image.

Google is rolling out a new Pixel Weather app that uses Gemini Nano to create AI-powered weather reports. The app also lets you customize how sections are arranged, so you can put information, such as the UV index, higher up in the grid.

Google will use AI to create summaries of phone calls after you hang up. Google says this is helpful for pulling out details from a call, citing the example of being referred to a new barber shop and forgetting to write down the phone number. The company says the calls and summaries are “never sent to the cloud” and that it notifies everybody on a call before it transcribes and summarizes it. Call Notes works on calls that are “at least 30 seconds long.”