Rewriting the Uber App in Swift
McLaren Stanley (via Steve Troughton-Smith):
The reason the Uber app is that large [331 MB] is because of Swift. The Objective-C version was a third of that size.
[…]
Library bundling is not the problem (swift system libraries are only a few MBs per slice). It’s the actual machine code growth rate of user code that’s the issue. Practically speaking Swift can be as much as 2-4x larger than the equivalent Objective-C when compiled.
[…]
We almost rewrote everything back in Objective-C. If not for the limit increase [in the cellular download limit] we probably would have.
Staying under the limit is really important, and they did rewrite the watchOS app back into Objective-C. Why is the app so large? Because there’s a lot more to it than first appears. On the other hand, the Android version of the app is much smaller.
Problem here with Swift came down to
- Apple not dogfooding its own tech. Hell they did not even help us or other major companies like AirBnB and LinkedIn (We formed a workforce on this problem with them which then forced the download limit to be upped by apple)
- Engineers deciding to adopt a language which they thought was great (and turned out not to be at the time) because they did not do the right analysis upfront for such a significant project. ‘Wow look at this shiny new cool tool, lets use it’ (Oversimplification, swift was more pleasant to write with and you could be more productive which was no doubt a factor here)
McLaren Stanley (Hacker News):
As a consequence of all these problems, there began to be a growing movement across all levels of the org that was rallying around the idea of “rewriting the app from scratch” The general sentiment was that the architecture was slowing us down, starting over would be faster.
[…]
On the iOS side of the world, the rewrite presented an opportunity to adopt Swift (which was in version 2.x during this timespan). Uber had tried Swift before, but like many who had adopted it that early on it was extremely problematic so it had been banned prior to the rewrite.
[…]
So this smaller core team of Design, Product, and Architecture went off in a room for with their new functional/reactive patterns, new language, and new app for a few months. Everything went well. The architecture relied heavily on the advanced language features of Swift.
[…]
But once Swift started to scale past ten engineers the wheels started coming off. The Swift compiler is still much slower than Objective-C to then but back then it was practically unusable. Build times went though the roof. Typeahead/debugging stopped working entirely.
There’s a video somewhere in one of our talks of an Uber engineer typing a single line statement in Xcode and then waiting 45 seconds for the letter to appear in the editor slowly, one-by-one.
Then we hit a wall with the dynamic linker. At the time you could only link Swift libraries dynamically. Unfortunately the linker executed in polynomial time so Apple’s recommend maximum number of libraries in a single binary was 6. We had 92 and counting.
[…]
We quickly discovered that putting all of our code in the main executable solved the linking problem at App start up. But as we all know, Swift conflates namespacing with frameworks; so to do so would take a huge code change involving countless namespace checks.
[…]
We also replaced all of our Swift structs with classes. Value types in general have a ton of overhead due to object flattening and the extra machine code needed for the copy behavior and auto-initializers etc.
[…]
So said brilliant engineer in Amsterdam, built an annealing algorithm in the release build to reorder the optimization passes in such a way to as minimize size. This shaved a whooping 11 mbs off the total machine code size and bought us enough runway to keep development going.
[…]
A bunch of folks burned out along the way. A ton of money was spent, hard lessons were learned, but still to this day most people insist the rewrite was all worth it. New engineers who joined up loved the architectural consistency and never knew the pain it took to get there.
There’s also this interesting bit on location data:
Without manual pickup location entry people’s location would just show up as whatever the GPS location was last received. This can be very inaccurate (especially in cities with tall buildings) and drivers would end up on the wrong block. This was a horrible customer experience.
So to improve location pickup we changed the location permission to collect signal in the background so we could send the drivers to your current location. People freaked out. Some of my ex-Twitter colleges called on me to quit such an evil company that would track you like this.
As a result of said freakout, (there’s a whole other thread about this involving @gruber and @TechCrunch that I’ll explain some other time) people turned off location permission. But the new app hadn’t designed any experience to handle this use case.
Previously:
- Rewriting Dropbox’s Sync Engine in Rust
- Rewriting Facebook Messenger
- Lessons From 6 Software Rewrite Stories
- Swift Compiler Not Optimized for Lots of Cores
- Uber Whistleblower on Location Privacy
- Rewriting Robotics Software in Swift
Update (2020-12-16): Indragie Karunaratne:
A tradeoff less discussed is in hiring: not long after Swift came out, the majority FB’s pipeline of iOS candidates wanted to write Swift in their interviews, and wanted to join a team where they could write Swift.
We had to figure out nice ways to tell them that they probably weren’t gonna end up writing Swift.
[…]
FB had already been struggling with binary size and build times long before Swift came out with their Objective-C++ codebase; it was a practical impossibility to adopt it, and probably still is even now, at least in the larger app codebases (FB, Messenger, etc.)
Previously:
11 Comments RSS · Twitter
This is an app to get a taxi. It's 331 Mb.
The Encyclopedia Britannica is 336 Mb.
https://data.nls.uk/data/digitised-collections/encyclopaedia-britannica/
If those 331 Mb are mostly code, and that's what he says they are, this is madness. The article is tagged "optimization". It should be tagged "slovenliness".
So a large company rewrites an entire app in a new language.../‘d that makes the Swifties who work there feeeel good.
The port costs a fortune, they need to implement a bunch of hacks to get the app to build. AND the app is essentially the same , in ObjC or Swift... and customers have no idea of the change...
BUT it was all worth it... because the Swifties feel goood.
SO management wasted all those resources to tinker with Apple’s new toy, to tickle themselves. pretty dumb.
"Swift" was quite the branding coup for a language that's drastically slower to compile, slower to link, slower to debug, slower to type ("waiting 45 seconds for the letter to appear in the editor slowly, one-by-one"), slower to learn, often slower to run, and results in final executables unavoidably bloated by 1.5-4x because "by [Apple's] own admission Swift will never compile as small as Objective-C".
> The article is tagged "optimization". It should be tagged "slovenliness".
Because people who use Uber often cross international territories, and may not want to have 3, 4, 10 different Uber apps on their phone, the app bundles in all options for all countries, for all Uber product possibilities, which adds up to 100s of screens. By comparison Britannica only has a two dozen at most.
A full breakdown is here: https://news.ycombinator.com/item?id=25376346
The Uber app offers an incredible user experience. It's Apple's Swift compiler team that dropped the ball by not providing an `-Os` option that could match Objective-C.
@OFL: Sticking with any Apple technology that isn't their newest up-and-coming one is a real risk. You can be using a technology that is still useful and sold to customers and actively updated, but if it isn't Apple's darling, your application could be in trouble.
You either start migrating early, when you can do it piecemeal, on your own schedule, and have the possibility of giving feedback to Apple about how to improve it, or you migrate later, when you're busy trying to ship a new release, and Apple surprises you by saying they're discontinuing some old tech you're using.
(Sure, SwiftUI sucks today, but is there any doubt it's going to be Apple's primary platform in 3 years, and UI developers using it are going to be able to code circles around plain UIKit developers? You can only afford to ignore it today if you trust Apple and your competitors to shake out all the issues you care about. From the linker problems alone, that's not the case for Uber. Uber screwed up *how* they migrated to Swift, but not *that* they migrated.)
Apple has never had a platform for people who think you avoid risk by sticking with the old way. That would be IBM: their latest Z systems are still compatible with applications written in the 1960's.
By comparison Britannica only has a two dozen [screens] at most.
For reference, the number I gave was for all text of all of the 143 volumes the Britannica Encyclopedia published between 1768 and 1860, i.e. one century's worth. Since 1860 predates computers, no screens were involved.
Even if Uber's app were to contain 100 different programs that did 100 different things, 331 Mb is preposterous. It's also very stupid. I need to access the internet to use Uber. If I have internet access, I can download a 5Mb app (300 / 60 = 5) when I need it over my cellphone connection. Most of the 300 million americans, or the 380 million Europeans, will never visit India, and will never need any support for an India specific payment system.
So yes, the tag should be slovenliness. Thanks for the link. It proves my point.
I guess people don't get how big 331 Mb is anymore.
The Coronavirus which has shut the world down is 30 Kilo base-pairs, i.e. 7.5 Kilobytes long.
The Hubble Extreme Deep field image encompassing 3000 galaxies is 13Mb. It wasn't that long ago that my computer struggled to load it.
A bird takes around 240 Mb. Is Uber's app more complex than a bird?
The entire human genome is 750Mb. Is Uber's app is half as complex as a person?
@Sam: you are right: Apple has never had a platform for people who think you avoid risk by sticking with the old way
Obviously this is a trade-off. Never changing anything means nothing can be improved. But rewriting everything constantly means you can't improve your product in other ways... and you can't learn the platform in depth: all its ins and outs, because it's always time for the new shiny thing. I'm no longer sure this trade-off makes sense: craftsmanship/technical debt matters little if it's all to be thrown away tomorrow.
It's surreal: many programs, such as R, don't run natively on Apple's M1 because it lacks a functional fortran compiler, and we still use LAPACK / BLAS even though it's 41 years old... https://developer.r-project.org/Blog/public/2020/11/02/will-r-work-on-apple-silicon/index.html Numerical code needs to be stable, to avoid arithmetic errors propagating indeterministically.
I'm curious about something, as a non-programmer. Is Swift always going to be this slow (etc)? Is there something intrinsic to how it works and compiles that means it's always gonna be slower and bigger? Or is it going to become, ugh, swift?
@Tony In terms of compiled code size, I think Swift will always be larger. Compilation speed will always be slower because it is so much more complex. Runtime speed is more complicated because there are areas where Swift is faster than Objective-C (that is not written in a C style) and other areas where Swift has a lot more overhead (e.g. sorting and iteration) vs. Objective-C without ARC. There is a long-term goal to address this by letting Swift programmers write certain sections of their programs at a lower level, but it does not seem to be a high priority at the moment.
@Sam wrote: "You either start migrating early, when you can do it piecemeal, on your own schedule, and have the possibility of giving feedback to Apple about how to improve it, or you migrate later, when you're busy trying to ship a new release, and Apple surprises you by saying they're discontinuing some old tech you're using."
Problem is nobody knows what technology will last. People who used the first versions of Swift started migrating early..."every line of ObjC is legacy code" some said. Many wrote apps in Swift as soon as it came out and now those apps won't compile.
Five-six years ago it was "use auto layout." Now it's "use SwiftUI..." so now...autolayout must be dead. Of course Apple has never as far as I know actually said these things explicitly...but this is what developers believe.
It's just not productive to rewrite an entire app in Swift that does the same thing as an existing ObjC version because you have a gut feeling Apple will stop supporting ObjC in 10 years. Ten years from now, nobody knows what Apple software platforms will look like, how much backward compatibility will be broken, and/or if the people working at Apple will already be pushing a *newer* UI framework that supposedly will be better than SwiftUI by that time.
