Mojo Programming Language
Jeremy Howard (via Hacker News):
But this “two-language” approach has serious downsides. For instance, AI models often have to be converted from Python into a faster implementation, such as ONNX or torchscript. But these deployment approaches can’t support all of Python’s features, so Python programmers have to learn to use a subset of the language that matches their deployment target. It’s very hard to profile or debug the deployment version of the code, and there’s no guarantee it will even run identically to the python version.
[…]
Unfortunately, Apple’s control of Swift has meant it hasn’t really had its time to shine outside of the cloistered Apple world. Chris led a time for a while at Google to try to move Swift out of its Apple comfort zone, to become a replacement for Python in AI model development. I was very excited about this project, but sadly it did not receive the support it needed from either Apple or from Google, and it was not ultimately successful.
[…]
So, if Swift was “syntax sugar for LLVM”, what’s “syntax sugar for MLIR”? The answer is: Mojo! Mojo is a brand new language that’s designed to take full advantage of MLIR. And also Mojo is Python.
[…]
A key trick in Mojo is that you can opt in at any time to a faster “mode” as a developer, by using “fn” instead of “def” to create your function. In this mode, you have to declare exactly what the type of every variable is, and as a result Mojo can create optimised machine code to implement your function. Furthermore, if you use “struct” instead of “class”, your attributes will be tightly packed into memory, such that they can even be used in data structures without chasing pointers around. These are the kinds of features that allow languages like C to be so fast, and now they’re accessible to Python programmers too – just by learning a tiny bit of new syntax.
[…]
As a compiled language, Mojo’s deployment story is basically the same as C. For instance, a program that includes a version of matmul written from scratch is around 100k.
There’s lots more information in the documentation.
That’s the trick with Mojo, our goal is not to make dynamic python magically fast. Yes, we are quite a bit faster at dynamic code (because we have compiler instead of an interpreter) but that isn’t by relying on a ‘sufficiently smart’ compiler to remove the dynamism, it is just because “compilers” instead of “interpreters”.
The reason Mojo is way way faster than Python is because it give programmers control over static behavior and makes it super easy to adopt incrementally where it makes sense. The key payoff of this is that the compilation process is quite simple, there are no JITs required, you get predictable and controllable performance, and you still get dynamism where you ask for it.
Mojo has a full ownership system that learned a lot from Rust and Swift and took the next step.
There’s also some discussion of what he thinks were mistakes made in Swift. It’s not clear to me whether the high level of sugar leading to performance problems was a case of not realizing the consequences until it was too late or of unsuccessfully gambling that they would find solutions—or whether they just thought the ergonomics were worth it. Regardless, Swift remains slow to compile and debug, despite the massive processor improvements since its introduction.
I enjoy writing code in Swift, but the tooling is disappointing. I’m growing weary of continuing problems with crashes and reliability (reporting incorrect errors and generating incorrect code). There are an ever increasing number of language enhancement proposals, most of which seem like good ideas in isolation, but the combination has made the language massive and complicated, with no end in sight. Even positing that this is the right path, I think it needs a few years with no changes other than bug fixes and perhaps optimizations.
Previously:
- Xcode 14.3
- High Performance Numeric Programming With Swift
- Swift for TensorFlow Canceled
- Chris Lattner on Swift, TensorFlow, MLIR, and SiFive
- Swift Ownership Manifesto
Update (2023-05-08): Damien Petrilli:
What it feels to me so far:
Swift: over engineered language which still fail to deliver its promises almost 10y later (broken toolchain / debugging, complex generics which doesn’t match the productivity of dynamic language, no bare metal perf)
Mojo: pragmatism end-to-end
[…]
My only beef with Mojo so far is that the syntax is not as clean as Swift.
Update (2023-05-15): See also: Hacker News.
Update (2023-05-30): See also: Hacker News (iainmerrick, brundolf).
Update (2023-06-07): Marcel Weiher (Hacker News):
Objective-C solved the two language problem by just jamming the two languages into one: Smalltalk for the scripting/integration and C for the component language. Interoperability is smooth and at the statement level, thougha there is some friction due to overlaps caused by integrating two existing languages that were not designed to be integrated.
Mojo essentially uses the Objective-C approach of jamming the two languages into one. Except it doesn’t repeat Objective-C’s mistake of using the component language as the base (which, inexplicably, Swift didn’t just repeat, but actually doubled down on by largely deprecating objects). The reason this is a mistake is that it turns out that the connection language is actually the more general one, the component language is a specialisation of the component language.
With this realisation, Mojo’s approach of making the connection language the base language make sense. In addition, the fact that the component language is a specialisation also means that you don’t actually need to jam a full second language into your base, a few syntactic markers to to indicate the specialisations are sufficient.
Update (2023-06-09): Lex Fridman:
Here’s my conversation with Chris Lattner (@clattner_llvm), a legendary engineer, his 3rd time on the podcast. We talk about Modular AI and Mojo, a new programming language that is a superset of Python and can achieve 35,000x+ speed ups over Python.
It makes your wonder if it the same guy who created Swift in the first place. I guess Chris learnt a lot from doing Swift.
