Sandbox Limitation on Number of Files That Can Be Opened
After loading a seemingly magic random number (around 3000/3200) of images the Sandbox will stop loading any more images. Any other operation that tries to load files from outside the container will fail.
The
NSOpenPanelbehave in a different way, it just won’t return more than the “magic random number” of images, and after that any attempt to use it will return zero files.If the files are loaded from inside the container, they will all load as expected, but after reaching the “magic random number” files from outside the container can’t be loaded anymore.
I don’t think I’ve blogged about this before, but I’ve heard many reports of it, and as far as I’m aware it’s a longstanding issue that dates to the introduction of the com.apple.security.files.user-selected.read-only entitlement in macOS 10.7.3. Note that this is not about the number of files that can be open simultaneously. The undocumented limit applies even if you close your file descriptors.
A related issue is that I’ve been using OmniOutliner a lot more recently, and after a while it will complain that it doesn’t have permission to save my document. Indeed, it doesn’t think the file even exists. I can neither save nor close the document without force quitting. The Console log makes it look like this is related to security-scoped bookmarks, which are used to access files that are saved in the Documents folder rather than in the application’s container. The problem dates to at least 2014 and also affects OmniPlan and Numbers, adding to the likelihood that the bug is in the OS rather than apps. None of the workarounds described in the preceding forum links worked for me.
Update (2018-02-05): Peter Steinberger:
Security scoped URLs have many gotchas.
Update (2018-02-13): I also ran into a problem where Downcast couldn’t access any of the files in its sandbox container because of problems with security-scoped bookmarks. I had to delete everything and reset it.
Update (2021-08-04): Sindre Sorhus (via Christian Tietze):
It’s just ridiculous how they could leave this broken for so many years. I have gotten many 1 star reviews because of this issue. I even have an alert in the app to inform the user about it. (Half the number of files, since it’s both open and save)
Update (2021-08-10): Michael Buckley (Hacker News):
There has recently been some renewed discussion on Twitter about the limitation that the macOS sandbox places on the number of open files an app can access at once. Apps are still running into problems because of this limitation, and there is not a lot of technical detail available on it outside of Apple, so I’d like to share my understanding of it.
macOS only allows sandboxed apps to access a limited number of files at a time, but there is no way for an app to query how many files it can open, or if it’s close to the limit. In fact, this limit is dependent on the amount of RAM installed in the computer and the number of files open by other apps.
I’ve heard others say that it’s not an “at a time” limit, but rather a total limit for the lifetime of the process, despite what the documentation for NSURL.stopAccessingSecurityScopedResource() says.
The engineer stayed late to help me file radars on the issue for both the security team and the fonts team, but cautioned that the underlying problem would probably not get fixed any time soon. It would be up to app developers and other teams within Apple to work around the problems the limitation created. It was foundational to how the macOS app sandbox was designed, and could only be fixed if they found a safe and performant way to store this mapping in userspace, which was unlikely. Apple was not going to throw out the sandbox or compromise its security to remove this limitation. Considering that it remains a problem to this day, he was right.
[…]
Considering that this limitation still affects high-profile apps like Microsoft Office, I don’t think Apple is unmotivated to fix this, but considering how long it has remained a problem, until Apple announces otherwise, we can’t assume it’s going away any time soon.
[…]
There are two ways to deal with the issue. The first is to relinquish security-scoped bookmarks as quickly as possible, and the second is to prompt users to open folders, not files.
If you go the folder route:
However, you may need to limit yourself to APIs that access those files using string paths, rather than
NSURLs.
This is the primary reason Retrobatch isn’t sandboxed (and thus, not on the App Store). I’ve considered just dealing with the likely support emails (and suggesting folks give it full disk access, or only add folder), but I just can’t bring myself to do it. What a pain this bug has been for years.
2 Comments RSS · Twitter
[…] Previously: Sandbox Limitation on Number of Files That Can Be Opened. […]
A big issue for developers is that only Applications can be entitled to use NSOpenPanel. If you program a plug-in like a screensaver that runs within a system application like the screensaver service (in Catalina now called "legacyScreenSaver" / the legacy in the name did not implied much hope for future support) there is no way the plug-in can be entitled. Unfortunately Apple Engineers decided not to give the service the rights to let the user select some files with the NSOpenPanel. The only entitlement this service now has is to access its own and the system cache folder(and only a temporary exception as the entitlement name suggests com.apple.security.temporary-exception.files. absolute-path.read-only).
I think it should be not a too big security concern to let a user decide that an screensaver can access at least to a bunch of pictures or movies that a screensaver can show. A developer of a screensaver is pretty much screwed today with Catalina and only very view developers try to create strange workarounds with helper applications that can be entitled and that copy files through the very small sandbox hole in the cache folders even if the entitlement of the system service is only a temporary exception and nothing to rely on for future updates. All this looks like the sandboxing feature was rushed and is still unfinished.
It is discussed here for example:
https://forums.developer.apple.com/thread/119008
https://forums.developer.apple.com/thread/117136
