Photos AI in appleOS 27
With Spatial Reframing, users can improve the composition of a photo after it’s been taken. Spatial Reframing builds on Apple’s deep understanding of spatial models thanks to Apple Vision Pro, so users can touch and drag a photo and preview in real time how the perspective shifts — as if they’d repositioned the camera in the original scene. Using powerful image models, Spatial Reframing will only generate new content where the perspective has been shifted, ensuring the reframed photo stays consistent with the original scene.
Users can also expand images with the Extend tool to give their subjects more breathing room. For example, they can straighten a crooked horizon without cropping out anything important, or adjust the aspect ratio, and Extend will fill in the missing pieces. Additionally, the popular Clean Up tool gets a major upgrade, so users can remove distractions with better quality and more realistic infill, even when the scene is complex.
These seem like useful features. I think it’s fine for Apple to offer them, but they do seem somewhat at odds with its previous preening about how they believe in the sanctity of photos that actually happened.
“Reframing photos with AI” - I don’t like this at all 👎
Kind of the crosses the line to cheating about reality too much for me… it’s one thing to lighten or darken or colorize pixels, but another thing to reposition them against each other.
“Deep respect for photography” right after a demo on generating images bc they do not respect art.
Generative AI is a technology that photographers are distancing themselves from (or should be), thanks to all the AI slop being produced everywhere. And yes, that includes creations from Apple’s Image Playground app, the image generator that the company also showed off during the WWDC keynote.
But generative AI doesn’t need to mean full images created from text prompts. When applied to selective areas, like erasing a piece of trash next to a subject’s feet, generative AI can do some of the menial work of replacing pixels that photographers would otherwise spend time retouching in an app like Photoshop. Google’s Pixel phones include a similar Magic Eraser tool.
Spatial Reframing is a great example of how the technology can be used to enhance real photos you capture.
I can personally confirm that Apple’s Photos “Clean Up” feature is DRASTICALLY improved in iOS 27. 😳
This is a bit hyperbolic and unkind, so I hope you can forgive me for venting. And, hey, who knows? Maybe the fancy new AI-powered reframing feature will actually get broad use. (But I doubt it…)
See also: MacRumors.
Previously:
- Apple Intelligence in appleOS 27
- Apple Intelligence in macOS 15.1 and iOS 18.1
- What Is a Photo?
- iPhone 16 Pro Camera
- Halide 2.15: Process Zero
Update (2026-06-18): Ben Lovejoy:
In the two weeks since, my view has been very much reinforced. The Clean Up tool is now significantly more powerful, while the Reframe and Extend tools address similar real-life needs …
7 Comments RSS · Twitter · Mastodon
The Ai reframing feature is exactly the kind of feature my kids and I will have fun trying to break by taxing the algorithm with the worst framing we can dream up.
This is yet another case where "AI" is tarnishing the name of a legitimately useful feature… for all intents and purposes, this one looks to be using Gaussian splats to reconstruct the scene for live editing. What percentage of an audience would give a standing ovation for this 10 years ago? How different is this to any other ML feature Apple already ships for Photos? Apple could have leaned into how cool the technology they are actually using is, but instead AI gets slapped on, with all its baggage.
I thought the "deep respect for photography" thing was always a bit stupid. Like, who cares? Don't value some abstract notion of purity above providing useful features to your users.
When I had a Google Pixel, I genuinely appreciated the Add Me feature, used it a lot, and miss it now. So good on Apple for shipping this, I hope they go even further in the future. If there's any worry, Apple could SynthID the more altered photos so people can still determine which ones are "real", for whatever that still means.
@Dimitri If so, it is indeed using generative AI models to create those splats. There are examples where it alters details already visible in the original framing: https://hachyderm.io/@samhenrigold/116720356522032713
@Plume Yeah, right now I think they are only using SynthID for fully generated images (Image Playground), but perhaps that should also be done when they are generating background content or infilling.
Apple VP Jon McCormack in 2024:
"Here's our view of what a photograph is. The way we like to think of it is that it's a personal celebration of something that really, actually happened."
The "deep respect" quote seems to be a continuation of that thread, but I can't reconcile it when generative imagery presents things that didn't happen.
I think they thought generative AI in 2024 was a fad, but it wasn't, now they don't want to admit they're 3 years behind, so you get this weird split-brain announcement.
@Hammer Yes, it comes across as their deep philosophical objection being more a rationalization because they couldn’t do it. Now they can do it, so they no longer see a conflict. Same with how they absolutely thought a chatbot was the wrong direction for Siri, but now there is a Siri chat interface and they have a pretzel explanation of how it’s not really a chatbot.
