Automating App Review
“The way to solve that inconsistency – and I hate to say it – is: let’s take a page from Google,” he told us. “Especially now with the AI tools that are out there. You can do probably 80% of the work the review team does.”
[…]
“Phil [Schiller] wants a set of eyes on every single app. I believe he is still basing that on one of the last things Steve [Jobs] told him, which is that you’ve always got to have a set of human eyes on every app that goes in the store. And Phil maybe carries that with him all the time.”
[…]
Shoemaker also questioned why app review guidelines have somehow become more vague over time. “The guidelines were written in a very grey way,“ says Shoemaker. “We wanted to have wiggle room to be able to shift our approvals or rejections…the idea was to start that way and then refine them over time.”
“They were rewritten in 2017 and they did none of that. In fact, they opened up more grey areas – it should be pretty solid right now, the guidelines should be very black and white.”
Via John Gruber (Mastodon):
App Store review times have decreased from an average of about 5 days to 1 day since Shoemaker left Apple in 2016. (And my understanding is that new automation tools are a big part of that process improvement. Shoemaker’s gripes about App Store review seem stuck in 2016.)
I think review times are not the main problem with App Review. But the average is definitely not the right way to look at them. It tells nothing about the likelihood of a maintenance update getting stuck in review for 61 days for no reason or of being blocked by a store bug for 54 days. Also not included in the average: apps and entitlement requests that never make it out of review but are just stalled indefinitely.
As someone who submits to both stores, I very much do not want Apple to automate the review process.
Google’s review process seems to run at random times, and will pull your app from the store for bizarre reasons. And when it does that there’s nobody at Google you can talk to about it.
I don’t think App Review does much good anyway - it’s incredibly easy for malicious apps to hide bad behavior from reviewers. And I know for a fact that they haven’t so much as looked at the paid portion of my app in more than a decade.
I think you could easily develop an automated system that would detect whether an app had changed enough to justify a human review, which would both give reviewers more time to focus on big stuff and let developers get bug fixes out faster.
Previously:
- Damus Zaps vs. the App Store
- App Store Competition
- App Store Director Developed “Animal Farts” and “iWiz”
Update (2023-06-28): Rob Jonson:
If you ever thought it was tough communicating with Apple, it’s a million times harder with Google. My apps have been pulled multiple times for the most stupid random (wrong) things. If I didn’t have friends in Google, things might never have been resolved. They spot some random technical infraction, then just remove your app entirely. Communication is just emails into the void with unthinking “computer says no” responses.
2 Comments RSS · Twitter · Mastodon
> What kind of AI does Shoemaker envision that's "consistent"?
ChatGPT is one implementation of one specific kind of AI. Deterministic AI also exists.
>“The way to solve that inconsistency – and I hate to say it – is: let’s take a page from Google,” he told us. “Especially now with the AI tools that are out there. You can do probably 80% of the work the review team does.”
What kind of AI does Shoemaker envision that's "consistent"? That's certainly not an adjective I'd use for ChatGPT.
>And when it does that there’s nobody at Google you can talk to about it.
Yup.
>it’s incredibly easy for malicious apps to hide bad behavior from reviewers.
This is tricky. How do you prevent an app from simply having a time bomb, after which its behavior changes?
>And I know for a fact that they haven’t so much as looked at the paid portion of my app in more than a decade.
Maybe, but as a counterpoint,
* we have an app for a hardware product of ours that does diagnostics, firmware updates, etc. App Review asked us to make a video showing us connecting the app to the hardware device. (And when we ported it from iOS to the Mac, App Review asked us for another video showing this working from the Mac as well.)
* and of course, there's the way to pass on login information for App Review. When we didn't, they asked us to amend it so they can try the app out more thoroughly.
So clearly, _someone_ there cares beyond "does it launch? Eh, then it's probably fine". This has been inconsistent and capricious over the years, and there have been countless examples of overreach (no, Apple, you don't need to check the entirety of Mastodon for porn), but if their goal is "reduce cases where apps are just scams that don't do what they purport to do", they're at least… trying?