Mojo Programming Language
Jeremy Howard (via Hacker News):
But this “two-language” approach has serious downsides. For instance, AI models often have to be converted from Python into a faster implementation, such as ONNX or torchscript. But these deployment approaches can’t support all of Python’s features, so Python programmers have to learn to use a subset of the language that matches their deployment target. It’s very hard to profile or debug the deployment version of the code, and there’s no guarantee it will even run identically to the python version.
[…]
Unfortunately, Apple’s control of Swift has meant it hasn’t really had its time to shine outside of the cloistered Apple world. Chris led a time for a while at Google to try to move Swift out of its Apple comfort zone, to become a replacement for Python in AI model development. I was very excited about this project, but sadly it did not receive the support it needed from either Apple or from Google, and it was not ultimately successful.
[…]
So, if Swift was “syntax sugar for LLVM”, what’s “syntax sugar for MLIR”? The answer is: Mojo! Mojo is a brand new language that’s designed to take full advantage of MLIR. And also Mojo is Python.
[…]
A key trick in Mojo is that you can opt in at any time to a faster “mode” as a developer, by using “fn” instead of “def” to create your function. In this mode, you have to declare exactly what the type of every variable is, and as a result Mojo can create optimised machine code to implement your function. Furthermore, if you use “struct” instead of “class”, your attributes will be tightly packed into memory, such that they can even be used in data structures without chasing pointers around. These are the kinds of features that allow languages like C to be so fast, and now they’re accessible to Python programmers too – just by learning a tiny bit of new syntax.
[…]
As a compiled language, Mojo’s deployment story is basically the same as C. For instance, a program that includes a version of matmul written from scratch is around 100k.
There’s lots more information in the documentation.
That’s the trick with Mojo, our goal is not to make dynamic python magically fast. Yes, we are quite a bit faster at dynamic code (because we have compiler instead of an interpreter) but that isn’t by relying on a ‘sufficiently smart’ compiler to remove the dynamism, it is just because “compilers” instead of “interpreters”.
The reason Mojo is way way faster than Python is because it give programmers control over static behavior and makes it super easy to adopt incrementally where it makes sense. The key payoff of this is that the compilation process is quite simple, there are no JITs required, you get predictable and controllable performance, and you still get dynamism where you ask for it.
Mojo has a full ownership system that learned a lot from Rust and Swift and took the next step.
There’s also some discussion of what he thinks were mistakes made in Swift. It’s not clear to me whether the high level of sugar leading to performance problems was a case of not realizing the consequences until it was too late or of unsuccessfully gambling that they would find solutions—or whether they just thought the ergonomics were worth it. Regardless, Swift remains slow to compile and debug, despite the massive processor improvements since its introduction.
I enjoy writing code in Swift, but the tooling is disappointing. I’m growing weary of continuing problems with crashes and reliability (reporting incorrect errors and generating incorrect code). There are an ever increasing number of language enhancement proposals, most of which seem like good ideas in isolation, but the combination has made the language massive and complicated, with no end in sight. Even positing that this is the right path, I think it needs a few years with no changes other than bug fixes and perhaps optimizations.
Previously:
- Xcode 14.3
- High Performance Numeric Programming With Swift
- Swift for TensorFlow Canceled
- Chris Lattner on Swift, TensorFlow, MLIR, and SiFive
- Swift Ownership Manifesto
Update (2023-05-08): Damien Petrilli:
What it feels to me so far:
Swift: over engineered language which still fail to deliver its promises almost 10y later (broken toolchain / debugging, complex generics which doesn’t match the productivity of dynamic language, no bare metal perf)
Mojo: pragmatism end-to-end
[…]
My only beef with Mojo so far is that the syntax is not as clean as Swift.
Update (2023-05-15): See also: Hacker News.
Update (2023-05-30): See also: Hacker News (iainmerrick, brundolf).
Update (2023-06-07): Marcel Weiher (Hacker News):
Objective-C solved the two language problem by just jamming the two languages into one: Smalltalk for the scripting/integration and C for the component language. Interoperability is smooth and at the statement level, thougha there is some friction due to overlaps caused by integrating two existing languages that were not designed to be integrated.
Mojo essentially uses the Objective-C approach of jamming the two languages into one. Except it doesn’t repeat Objective-C’s mistake of using the component language as the base (which, inexplicably, Swift didn’t just repeat, but actually doubled down on by largely deprecating objects). The reason this is a mistake is that it turns out that the connection language is actually the more general one, the component language is a specialisation of the component language.
With this realisation, Mojo’s approach of making the connection language the base language make sense. In addition, the fact that the component language is a specialisation also means that you don’t actually need to jam a full second language into your base, a few syntactic markers to to indicate the specialisations are sufficient.
Update (2023-06-09): Lex Fridman:
Here’s my conversation with Chris Lattner (@clattner_llvm), a legendary engineer, his 3rd time on the podcast. We talk about Modular AI and Mojo, a new programming language that is a superset of Python and can achieve 35,000x+ speed ups over Python.
It makes your wonder if it the same guy who created Swift in the first place. I guess Chris learnt a lot from doing Swift.
6 Comments RSS · Twitter · Mastodon
I have in my drafts a list of "thoughts", in the style of my earlier post on Swift, as I thought this would be a good thing to reopen my blog with. But even limiting myself to preconcurrency Swift, I keep postponing it because I feel like Swift is not getting any closer to maturity. Case in point: this fix https://github.com/apple/swift/pull/41826 , which I read as "oops, we failed to consider the possibility that a closure could be recursive but non-escaping, who could possibly have guessed?"
That being said, when my disappointment in Swift's performance led me to consider using a particular algorithm as an opportunity to learn a new language by rewriting it in Rust, the performance ended up being similar because of Rc and RefCell overhead, and the turnaround time ended up being the same as well (if you think the Swift inference engine is overly complex, pray you never have to deal with Rust's borrow checker).
And I feel the same about this new development, so far. In particular, they propose a Python superset that affords ways to constrain its semantics when needed (as opposed to similar projects that implement a subset of its syntax as a way to obtain that), but the assumption of dynamism is embedded in the most fundamental Python constructs, so there is bound to be disappointment somewhere. For instance, Python supports optional parameters even for anonymous functions, which means function parameters are bound dynamically…
Allowing users to declare function argument types and having the compiler use those declarations for optimization made a big performance improvement in MACLISP back in the 1960s. I guess it was a good idea then, and it's a good idea now. All things old are new again.
I respect those who are devoted to the language, but Swift has become too complex. It feels like it's becoming just another C++.
I want Objective-C without the C, but I feel Apple is too deeply malinvested into Swift now. They never admit mistakes, nor will the Swiftfluencers and Apple sycophants who pushed us down this hole. Now we're stuck with a language that imo is a net-negative compared to Obj-C. Swift has some nice features, but complexity, slow builds, buggy tools, and a push to code xib files by hand is the opposite of what I want. Drastically cutting features to tame complexity is what I want. That's what Objective-C without the C looks like to me.
I'm glad Lattner is opening up about Swift. Learning from mistakes makes Mojo's story interesting, while the alleged performance gains make it worth watching. We need more honest discussions about the state of Swift, instead of 9 more years of "safe code" chants and parades over esoteric features for language academics.
Is it too early to talk about what replaces Swift?
Is Mojo the kind of language that could be used for macOS instead of Swift? Hypothetically speaking, of course. Is it a better language than Swift?
Thank you, I'd miss this announcement. Feels like Python finally getting its Typescript.
As for the state of Swift, I 100 % agree, and I would love if the team created some opt-in mode that would get rid of that bits of syntax sugar that make Swift slow to compile. I'd rewrite some parts of my apps in a few days and never look back.