Every new iOS 18 AI feature for the new Siri just leaked

We’ve heard plenty of rumors about Apple’s alleged plans to super-power its digital assistant Siri at the 2024 Worldwide Developers Conference (WWDC). Today, via Apple Insider, we have the most comprehensive leak yet. Sourced from “people familiar with Apple’s AI initiative,” the leak contains pretty much everything Siri will be able to do across over a dozen first-party iPhone apps.

The full leak is worth a look, especially if you’re an iPhone user. However, we’ll give you the general gist of Apple’s goals with the “new” Siri and share some highlights that we think will most affect iPhone users’ day-to-day lives.

What Apple wants from the ‘new’ Siri

The overall goal for Siri appears to be making it more powerful and better at understanding voice commands that are delivered in natural language. According to the leak, Apple has allegedly been training Siri for this by having Apple technicians deliver commands that are purposefully obtuse. For example, instead of asking something like, “Hey Siri, show me pictures of my cat,” it is testing vaguer commands like, “I want to make a blog,” or, “I am feeling nostalgic right now.” These are not specific commands instructing Siri to do one particular thing, but instead commands Siri will need first to interpret and then decide how best to deliver what it thinks the user might want/need.

The advantage of this is obvious, which is training Siri to be better for users who don’t know (or don’t want to use) the proper syntax needed to execute a command. For example, a user saying, “Hey Siri, I want some coffee,” may or may not turn on the smart coffee machine, while saying, “Hey Siri, turn on the coffee machine,” likely would. The former is a natural statement, while the latter is a direct command. Apple wants this locked-in syntax reduced, making Siri much easier to use.

The Apple Insider leak doesn’t mention how this works, though. For example, are these Siri features powered by “Ajax,” which is the codename for Apple’s internal large language model (LLM)? Or are these based on ChatGPT, since Apple has allegedly partnered with OpenAI for some of its AI-based systems? It might be a little of both, but we’re not sure yet.

New Siri features: A list of highlights

As mentioned, the full leak is exhaustive, going over no less than 18 first-party apps for the iPhone and how Siri will be able to work with each one. Here are a few that we think are really interesting:

• Camera: Siri will be able to control the camera through voice commands. You’ll be able to toggle video recording on or off, open the camera in a specific mode (photo, portrait, video, etc.) and then start a shutter timer, and flip to the front or rear camera. Theoretically, this could allow you to set your iPhone up for a group photo, walk away, and then use voice commands to capture the photo remotely.
• Mail: The Mail app is getting a complete overhaul. It will apparently be able to automatically classify email using machine learning, something with which Gmail users are likely already familiar. On top of this, Siri will also be able to perform detailed functions through only voice commands. This includes things like composing an email, sending it, scheduling it, marking an email as junk, and setting a reminder to read an email at a later time. It will also be able to summarize emails and create “smart replies,” a feature undoubtedly just like Smart Reply on Android and Help Me Write in Gmail.
• Photos: Apple is likely to introduce a lot of photo editing features based on generative AI. Pixel users will likely recognize a lot of these, as we’ve only heard so far about features you can already do on Pixels with Magic Editor and Google Photos, such as move/remove an object from the photo and fill in the blanks with generative AI, find specific photos with specific people/animals, and apply generative AI filters.
• Safari: Apple’s web browser will use Siri for webpage summaries, something Google has already brought to Android through Gemini. Safari will also be able to create new tab groups or open a new Private tab through voice commands.
• Voice Memos: You’ll be able to go completely hands-free with Siri using Voice Memos. For example, you could ask Siri to create a new voice recording and then start talking. You could then stop the recording, save it with a specific name, and then even move it to a specific folder — all without laying a finger on your iPhone.

When will these launch?

According to serial Apple leaker Mark Gurman, at least some Siri features won’t actually land at WWDC. Apple will almost certainly announce a few of them, but not all will be available in 2024. Gurman posits that it will be 2025 before the majority of these features land through a software update.

Of course, that doesn’t mean Apple won’t roll out at least a few at or around WWDC. However, it’s likely best not to expect that iOS 18 will come with all the features in this Siri leak, as it’s far more probable that these will drip out over the coming months.