Scripting Languages to Be Removed
Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. In future versions of macOS, scripting language runtimes won’t be available by default, and may require you to install an additional package. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app.
[…]
Use of Python 2.7 isn’t recommended. This version is included in macOS for compatibility with legacy software. Future versions of macOS won’t include Python 2.7. Instead, it’s recommended that you run python3 in Terminal.
This is a big deal in terms of philosophy; Apple once touted the built-in Unix tool suite as a Mac advantage. And it also means lots of practical changes; installers and AppleScripts can no longer lean on other scripting languages.
The same year Microsoft is proud to add a new terminal app with the full support of bash and everything to please developers. Apple decided to deprecate scripting on macOS and replace bash with zsh.
No explanation was given, but we’re curious. Maybe there’s a valid reason.
The downside of all that stuff getting included was that it could never be upgraded, because apps would depend on the exact versions that shipped a decade ago.
It still doesn’t make sense to me because these scripting languages aren’t huge, and Apple did update them and remove old versions from time-to-time.
Previously:
Update (2019-06-04): Adam Maxwell:
I guess the kids all think this is a good change, because now no one will have an old version of Python or Perl. Instead, they’ll have ten of them, and you’ll have to play with paths to pick the right one, like Windows.
Update (2019-06-06): See also: Lobsters, Reddit, Hacker News.
Update (2019-06-10): Dr. Drang:
One of the great values of Apple providing scripting languages with macOS is to make it easy for regular users to turn themselves into occasional scripters. People who program all the time seem to forget how terrifying the idea of writing code is to those who’ve never done it before. Even “power users” quail at the idea of scripting until they’ve done it a few times and survived. If the first step in learning to script is Install Python via Homebrew, they are simply going to pass.
These new scripters don’t care, for example, that the Apple-supplied Python is out of date, and there’s no reason for them to. Python 2.7 is a perfectly good starting point, as is Perl 5.18 and Ruby 2.3.
Removing scripting languages also means that those of us who know how to use them can’t send scripts to our friends to help them automate their tasks. I’m perfectly happy to “slum” in Python 2 (oh dear, I have to use the format function instead of f-strings) if that’s what it takes.
Microsoft (via Hacker News):
Microsoft has been involved with the Python community for over twelve years, and currently employ four of the key contributors to the language and primary runtime. The growth of Python has been incredible, as it finds homes among data scientists, web developers, system administrators, and students, and roughly half of this work is already happening on Windows.
[…]
Finally, with the May 2019 Windows Update, we are completing the picture. While Python continues to remain completely independent from the operating system, every install of Windows will include
pythonandpython3commands that take you directly to the Python store page.
Update (2019-06-12): Kyle Crawford:
Scripting runtimes and bridges should be included in an OS. I like Swift. I’m comfortable with Swift. But I also like ruby and python. And often they are a better fit for the job. I’ve written a lot of Swift but I still write lots of ruby and python.
Update (2019-06-27): Curtis Wilcox:
Although multiple apps needing their own copies of Perl, for instance, might seem inefficient, developers need to include only the runtime and exact libraries they use, not a full scripting language installation. Some apps have already taken this route to keep using Java, and many modern apps rely on the Electron framework, which includes Node to run JavaScript.
If you look at the sizes of the different parts of each installation, I don’t see how shipping separate copies of the runtime would ever come out ahead. The Electron framework in a single app is larger than the entire system-provided Perl.
Considering how integral these runtimes are to my own daily use of my Mac, this is a real annoyance. I’m fully capable of installing them myself, but I still find the decision baffling.
[…]
So while this might not be a huge blow for me, it’s an impediment to anyone who uses my projects that isn’t a Terminal nerd.
[…]
So the question is why? I can’t wrap my head around any real benefit to Apple’s line of reasoning on this. They’ve not been great about keeping the runtimes up to date, but that’s actually been a boon, requiring less effort to keep scripts working with every OS release. These runtimes were even touted as a selling point in the past, and Microsoft is just now starting to add tools like this to the default Windows install. And they’re including a sexy Terminal. So why is Apple moving in the opposite direction?
I’m one of those people who just use whatever’s on the system.
Me, too.
Something that only just occurred to me is this will break every single script with
#!/usr/bin/perlirretrievably since /usr/bin is a SIP protected directory. Every one will need to be edited.
Having to install Python/Ruby/etc. manually is gate-keeping. It might be easy for pros, but it’s not easy for everybody.
Are we going to ask that person to figure out how to install Ruby? If there are one-click installers out there, are we going to ask them to figure out which one is actually reputable and safe?
Curiosity like this is one of the ways new developers are made. I worry that the less the Mac is tinkerable out-of-the-box, the fewer developers we’ll get.
But something about this feels both arbitrary and inherently wrong. The beautiful thing about MacOS is that there’s a visually coherent interaction layer that most users spend most of their time in, but anyone — including a lunkhead like me — can fire up the Terminal at any time and run a script. Having that capability at one’s fingertips just below the surface, as well as programs like Homebrew and MacPorts, makes the Mac feel limitless.
28 Comments RSS · Twitter
I'm dismayed by this, as I use Python in TeX Live Utility, and hate the thought of porting that to another language or trying to bundle an interpreter. TeX Live uses Perl heavily, so that will require bundling Perl, just like Windows. Even worse, all users will end up with multiple Perls and Pythons to confuse the hell out of everything. It's a shame to see the Mac turn into Windows from a decade ago, when it used to be great for UNIX stuff.
It also makes me wonder what happens to the bridge support; remember when PyObjC was a big deal? I still have some scripts that use it for image processing and other things. I also wonder what happens to conventions like /Library/Python, and module support? Man, I hate Apple these days.
For better or worse, most Python projects run inside virtual environments where you're controlling the specific version of everything - Python and related packages.
This reminds me of the early years of Mac OS X where it was required to install an additional package (BSD additional or something like this) to get some classic command line tools included (e.g. grep).
I can understand this with at least Python given the differences between Python 2.7 and Python 3.x. Apple's following their own advice. Apparently Xcode 11's debugger switches from 2.7 to 3. For those not familiar with it you can have Python scripts in the debugger and do lots of cool things. (I actually wrote a script that put my watch variables into a Numbers spreadsheet that updated with each breakpoint)
While this sucks, the reality is that with Python as Lee notes run in virtual environments, partially due to all the different versions running around in macOS or other OSes. Embedding a version of Python, Perl or other interpreters gives you a lot more stability with what's going on. If you're calling shell scripts with Applescript this is a bit more of a pain, but it's not hard to have a standard Homebrew or MacPorts install if you are doing that sort of thing.
>This reminds me of the early years of Mac OS X where it was required to install an additional package (BSD additional or something like this) to get some classic command line tools included (e.g. grep).
That separate installer was for the same reason as these final removals: fear of GPL. By making the "BSD" package an optional install, the lawyers felt it kept them clear of any confusion. Since GPLv3, the ambiguity was removed and any presence was an explicit virus, so Apple stopped upgrading any of them, which lead to a perpetual slow-motion usability & security fiasco. Apple's simply giving up on pretending to be a full fledged UNIX, and putting security first.
@Jordan But, to be clear, Perl, Python, and Ruby don’t use the GPL. Though that probably explains the bash/zsh situation.
This kind of sucks but I think most people who rely on scripting languages on macOS are already installing up-to-date versions of them through Homebrew or something similar.
I'm curious how "Just use homebrew" would work if you need Ruby to install the homebrwe? You would have to bootstrap Ruby, and then install homebrew, and only then install whatever else is needed.
I'm not sure if rbenv/rvm can work easily on a "bare" system, which means you need to build the ruby from source?
Funny enough, that "bare" macOS feels more like FreeBSD or an old-school linux.
@Adam Maxwell: “It also makes me wonder what happens to the bridge support; remember when PyObjC was a big deal?”
PyObjC dates from NeXT so has never used .bridgesupport. AppleScript-ObjC does use it, so as long as Apple officially “supports” AppleScript they should keep it working, caveat various bits of it breaking whenever the Swifties mess with the ObjC headers.
I’ve got a feeling gen_bridge_metadata was written by Laurent Sansonetti, presumably as part of his MacRuby project, but he left Apple quite a few years ago so I doubt has anyone within Apple sponsoring it now. The same can be said for macOS’s bundled Perl/Python/Ruby/Lua/Tcl support; I think a lot of that was just individual engineers’ pet projects, not a company strategy.
This doesn’t prevent third-party bridges implementing their their own glue generators a-la PyObjC, of course. It was just convenient to have a single common standard that worked out of the box. (Again, caveat bits breaking; I don’t think Ronald Oussoren [longtime PyObjC maintainer] thought too highly of it, but you’d need to ask him why.)
..
Also bear in mind that the future of macOS &co lies with Swift, and bridging to that is different problem entirely. And that’s before we even consider sandboxing, distributed computing and other essential modern requirements. I doubt traditional *nix-y scripting languages will adopt those soon, if ever. They’re really last-generation languages; they can always slap more shims on top, but at some point it makes far more sense to draw a line under them and start afresh.
It’s clear that Apple see the standardized, productized, and hopefully widely accessible future of *OS scripting and automation in Siri Shortcuts. It’s also clear to those of us who’ve been around the block more than once that Shortcuts misses a huge opportunity to support freeform fine-grained text- and voice-based input as easily as GUI, but that could still be added by replacing its opaque XML serialization format with a proper first-class Lisp-like language.
I’ve spent the last 10 years learning to design, build, and train end-user automation DSLs and have already had a couple runs at POC-ing a new-generation of general-purpose scripting language built from ground-up in and for the Swift ecosystem, so am starting to get a picture of what’s needed and how to make it.
It’s even-odds if I have a third crack with a view to an actual deliverable—I need a project that’s going to pay as I can’t work for air any longer, and right now the #1 contender (cross-platform artwork automation) is something that’ll take me away from the Swift ecosystem entirely. Ditto on my recent work towards a pure-Swift reimplementation of Apple events. I can see true greatness as one future, but I also know what a huge logistical ball-ache getting there would be, and it’ll never make me wealthy like some of my other projects could. So we’ll see.
@Fred McCann: “This kind of sucks but I think most people who rely on scripting languages on macOS are already installing up-to-date versions of them through Homebrew or something similar.”
Indeed. I’m absolutely fine with move myself; macOS, like every other *nix, is an absolute trash house, and desperately need of some product-oriented focus and clarity. I’m just a bit disappointed they’re not being a lot more aggressive; I mean, when’s the last time anyone here used groff/troff, and just how many alternate shells (csh, ksh, tcsh,…) does macOS really need to include as standard? Huge unkempt surface area—not at all conducive to shaping a secure mass-market OS.
Again, Apple could solve both problems and make everyone happy by picking out the best of the third-party package managers and punting the problem over to that as an officially blessed solution, with a modest allocation of resources to ensure its managing body is and stays on a solid track. (A good investment given the corresponding reduction in size—and thus security, maintenance, and support costs—of their own OS distro.) Feel free to start a Radar ticketing campaign towards that end.
I think Apple for legal reasons wants to keep some distance between itself and MacPorts or Homebrew. It's a bit frustrating that each repository tends to have different apps, although the main big stuff are in both. On the other hand having two means that there's a bit of choice and backup. I can't see Apple really feeling the need to bless either given that both for the most part work quite well. Those who need the shell can just use the repositories and then all the GPL nonsense isn't an issue for Apple.
I've not kept up with Window's embrace of Linux like services. I wonder how they're balancing the GPL legal issues.
If GPLv3 is such a concern to Apple, then we know what’s coming next. GPLv3 was specifically created to address “Tivoization”: providing the source code as required by the GPL, but locking down the hardware in such a way that the user can’t actually install modified software.
GPLv3 in newer bash versions shouldn’t be an issue unless Apple is planning on locking MacOS down further.
@Clark
> I've not kept up with Window's embrace of Linux like services. I wonder how they're balancing the GPL legal issues.
Presumably it isn’t a concern because they aren’t planning on locking Windows down a la iOS.
LLDB: Beyond "po" (Session 429)
"[…] Discover […] how to extend LLDB using your own Python 3 scripts."
I guess the LLDB team did not receive the memo.
@Clark: What legal issues do you think might be at play? It may be that a PM project needs to establish a legal organization, e.g. a-la PSF, to ensure correct administration and decoupling of concerns between its own software, the hosted code, project management, and its sponsors. But that’s why we have things like lawyers and accountants; and an organization like Apple could easily fund both the setup and ongoing running costs probably for less than the cost of an in-house engineer while being free of legal encumberment.
Just in terms of messaging alone, I’d say that was money well-spent. It generates user confidence that this software will continue to be well supported and up-to-date on Macs, while acknowledging that its user community are the ones best equipped to do this. Let’s face it; /System and /Library are where third-party software products go to die. This hands control back, along with a very nice ego fluff. It’s the best deal that little folks can hope for while dancing in the same room as the 80,000lb gorilla.
Also, as I say, once everyone who needs to use this stuff is used to using PMs, that makes it much easier for Apple to offload further chunks of the old BSD userland which, being completely useless to 99% of users, represents nothing but risk and cost for both Apple and that 99% (and, arguably, for the 1% too).
Keeping the geeks sweet is just as important as keeping control of the development platform: both are necessary [evils] to generating Apps, which generate revenue, which make corporate and shareholders very happy indeed. If sponsoring a PM can make those code-munchers feel super special without having nanny their asses <em<and gets shot of a load of irrelevant legacy code, that’s an excellent 2-for-1 bargain for Apple too.
@Will Notbepublished: “I guess the LLDB team did not receive the memo.”
Last time I looked, LLDB wasn’t a stock install either.
Does installing the optional Command Line Tools package (either via Xcode or directly from Terminal) automatically install Python 3 along with any other third-party dependencies it requires? If not, go file a Radar on that.
A lot has changed in terms of managing scripting language versions in the last 10 years. FWIW any time I ever relied on the default installation of a scripting language on OS X I came to regret it.
Has, I obviously don't know. I suspect getting burned in the past with things like zfs means they're a bit paranoid over open source - perhaps more than many people think they ought to be. But from their perspective it's better to be paranoid now rather than lose IP or have to make emergency course changes because of some unexpected lawsuit. Not being an attorney I don't know. I know most geeks think there's no issue but most people aren't attorney's representing a multi-billion dollar company that many open source proponents don't like.
[…] “Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. In future versions of macOS, scripting language runtimes won’t be available by default, and may require you to install an additional package.” [From the XCode 11 beta release notes | Commentary on Michael Tsai’s blog] […]
[…] guess that’s why there won’t be room for scripting languages. But, seriously, I wonder why they can’t be generated […]
@Clark
Wait a minute now, I don't think the ZFS license provided any problems for Apple. Was that not licensed under CDDL? A lawsuit, regarding software patents (boo!) showed up later, but any company can be sued over software patents. Apple had been sued, and lost in fact, with their own home built technologies. VirnetX has beaten Apple in court multiple times now over FaceTime infringement.
Also, @Drunken Dogcow has it correct as far as I can tell. Bundling a GPLv3 app on your system does not mean your whole system becomes GPLv3, just modifications and such to those specific apps. No? I mean, my Linux kernel is GPLv2 but I install many different apps with many different licenses on top of the kernel. Installing Samba doesn't suddenly make everything GPLv3! Not the Linux kernel, not closed source games, nothing.
@Jordan
@Michael Tsai is correct, GPL3 boogeyman has no bearing on at least three of the major languages involved.
Honestly, I think Apple is incredibly lazy with updating their command line tools and it is probably for the best they stopped bundling incredibly old tools without security patches and bug fixes.
P.s. At to bash being a problem because it is GPLv3, how come this problem does not afflict Linux distributions which contain quite a bit of GPLv2 and LGPL tools? Or now Windows for that matter?
@has
OK, let's then look at the terrible Feedback Assistant.app application in Catalina betas:
It contains a python script and 2 bash scripts.
[…] At the lowest levels you’ve got full access to a Unix environment with Bash (haha) and all the other standard scripting languages and tools. (For now.) […]
If Apple continues to "evolve" macOS in the direction that they do now, its operating system will be removed from my computers.
I'm honestly happier to use Windows 10 today than the recent release of macOS.
I was going to use applescript to change system preferences... But at this point, instead of figuring it out, should I learn a different language like Swift or Java?Automator is on the way out and too limited.
[…] scripted in Python, Ruby or perl using their JSON modules. The problem is, those runtimes have been slated for removal from a future macOS. So I took that as challenge to devise a method to query and modify JSON data […]
[…] Extract the code using a shell script. (Please dear god Apple don't remove Perl from Monterey.) […]
