{"id":20483,"date":"2018-02-13T01:00:37","date_gmt":"2018-02-13T06:00:37","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=20483"},"modified":"2021-09-06T10:20:50","modified_gmt":"2021-09-06T14:20:50","slug":"how-apple-plans-to-root-out-bugs","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2018\/02\/13\/how-apple-plans-to-root-out-bugs\/","title":{"rendered":"How Apple Plans to Root Out Bugs"},"content":{"rendered":"

Mark Gurman<\/a> (tweet<\/a>, MacRumors<\/a>, Reddit<\/a>, Hacker News<\/a>, ArsTechnica<\/a>):<\/p>\n

\n

Instead of keeping engineers on a relentless annual schedule and cramming features into a single update, Apple will start focusing on the next two years of updates for its iPhone and iPad operating system, according to people familiar with the change. The company will continue to update its software annually, but internally engineers will have more discretion to push back features that aren’t as polished to the following year.<\/p>

[…]<\/p>

The shift is an admission of what many customers have already come to notice: Some Apple software has become prone to bugs and underdeveloped features. […] Apple has also recently released features later than it expected, as the rush to meet the annual deadline overtaxed engineers and created last-minute delays.<\/p>\n<\/blockquote>\n\n

John Gruber<\/a>:<\/p>\n

I’m not so sure the above is a new strategy so much as a tacit admission of what’s actually been going on the last few years.<\/p><\/blockquote>\n\n

Then why should we expect any improvement?<\/p>\n\n

Jeff Johnson<\/a>:<\/p>\n

The idea of postponing features a year until they’re “ready” misses the whole point. It’s very difficult to find all the bugs in a major change until after you ship it. To get to a stable operating system, you need to spend at least a year just fixing bugs after a major release.<\/p><\/blockquote>\n\n

You can’t just consider the internal costs of annual updates. There are major external costs. Third party developers play an essential role in QA. If we never get the thing until June, and you ship every fall, never enough time to fix bugs.<\/p><\/blockquote>\n\n

A lot<\/a> of<\/a> people<\/a> are pointing to Steven Sinofsky’s comments<\/a> (Reddit<\/a>). He makes some good points about the “broader context,” but I think he’s completely wrong about Apple’s software quality:<\/p>\n\n

In any absolute sense the quality of Mac\/iOS + h\/w are at quality levels our industry has just not seen before. […] On any absolute scale number of bugs—non-working, data losing, hanging mistakes—in iOS\/Mac is far far less today than ever before.<\/p><\/blockquote>\n\n

I don’t see how that can be taken seriously. He doesn’t have access to Apple’s bug database, so how would he know? I really doubt that the number of open bugs is lower than in the past, and even if it were there’s no reason to assume that Radar is representative of the actual number of bugs. He later says that the list of bugs is “infinitely long,” so this whole line of argument seems nonsensical. In what way is today’s Mac\/iOS quality better in “any absolute sense” than in, say, 2010? He doesn’t say, except that more people are using it:<\/p>\n\n

What is different is that at scale a bug that happens to 0.01% of people is a lot of people. A stadium full or more. […] No one ever anywhere has delivered a general purpose piece of S\/W+H\/W at scale of 1B delivering such a broad, robust, consistent experience. We don’t have a measure for what it means to be “high quality”.<\/p><\/blockquote>\n\n

Well, we can look at how many problems an individual user runs into. Is it higher or lower than before? This measure is independent of Apple’s scale. So is the circle of people I hear complaining. Apple’s customer base has doubled many times over, but the number of family members, friends, and customers that I communicate with has not. Now you could argue that maybe we have become exceptionally unlucky and are running into more than our share of issues, but I don’t find that very convincing.<\/p>\n\n

He wants to discount the actual experiences of “many super smart\/skilled people” because “the more a product is used the more hyper-sensitive people get to how it works.” What does that even mean? The number of hours in a day hasn’t increased; I don’t think my Mac\/iPhone usage has increased much, if at all. Hardly anyone complains to me about the “slightest changes”; I hear about things that flat out don’t work. That’s not being hyper-sensitive.<\/p>\n\n

Previously: Apple Delays Features to Focus on Reliability, Performance.<\/a><\/p>\n\n

Update (2018-02-13): jarjoura<\/a>:<\/p>\n

\n

As someone who used to work on iOS at Apple, what that company honestly needs is a culture not beholden to the whims of their EPMs (project managers). They used to help organize and work with engineering to schedule things across the company’s waterfall style development. However, by the time I left, they essentially took power over engineering. Radar became the driver for the entire company and instead of thinking about a holistic product, everything became a priority number. P0 meant, emergency fix immediately, P4 meant nice to have. You get the idea.<\/p>\n

Nothing could be worked on if it wasn’t in Radar with a priority number attached and signed off by the teams’ EPM. No room for a side project or time away from your daily duties because there were always P1s to fix. If you didn’t personally have any left for the day, you’d take one from another engineer who was likely swamped with their own list of P1s.<\/p>\n

[…]<\/p>\n

This is how you get bugs in shipping software. EPMs driven to schedule things and over manage engineers would decide on a whim that something was a P2. That was basically always shelved to a follow-up .1 release.<\/p>\n

Ultimately, engineers lost the freedom to decide when a feature was ready to ship.<\/p>\n<\/blockquote>\n\n

This point about bug prioritization came up two years ago<\/a>.<\/p>\n\n

Bob Burrough<\/a>:<\/p>\n

This is absolutely, 100% true, and jibes with my experience.<\/p>

There was (don’t know if there still is), another really whacky problem with iOS work prioritization back then. Radar has P1, P2, P3, etc priorities. Milestones were arranged such that “No P3’s” happened (the point at which P3’s were no longer allowed to be worked on)… followed by “No P2’s” then finally “No P1’s.” At first glance that arrangement makes sense because it means the only bugs getting fixed late in the game are the really high priority ones. However, what it meant in practice was, if there was a P2 bug an engineer wanted to fix… They would scramble to make sure it gets fixed before the “No P2’s” milestone occurs…in effect, causing P2 bugs to be worked on before P1’s.<\/p><\/blockquote>\n\n

throwaway7187<\/a>:<\/p>\n

I’m a former iOS EPM (not speaking for Apple, obviously, since I don’t work there anymore), and although the Reddit commenter got the atmosphere of constant crisis right, he\/she is misplacing the blame and misunderstanding the power dynamic. EPMs at Apple essentially have zero power over engineers’ workload. They take the list of stuff the engineering managers said they want to get done this year and say “You guys are crazy, you’ll never be able to do this without 3x the hours\/manpower.” Then they proceed to drive the team as hard as necessary to make sure that they actually deliver what they said they were going to deliver. That’s it. The idea that there is this cabal of mighty EPMs twirling their mustaches and loading developers down with work is pretty far from reality.<\/p>

It’s true that you shouldn’t be working on anything not in Radar (the bug tracker) but this is true anywhere you’ll work. Project managers however do not sign developers up for all those radars--on the contrary--we’re usually trying desperately to help you get rid of scope and get the task list down to what’s actually do-able!<\/p>

One of the great things that IMHO sets Apple apart is how engineering-driven they are. I’ve never worked anywhere else where engineers had so much freedom to decide what they’re working on. The fact that they always decide to work on 3x what they can actually achieve is kind of on them. But that drive to try to do so much is part of what keeps innovation strong at Apple.<\/p><\/blockquote>\n\n

Benjamin Mayo<\/a>:<\/p>\n

\n

It sure looks like this is a case of the feedback loop working. The Apple community complains about software quality, the executive team reviews procedures and makes structural changes.<\/p>\n

[…]<\/p>\n

As an outsider, I think it’s hard to really assess whether these changes are meaningful rather than empty, ambitious, words. However, I’m glad the way it is portrayed in the Bloomberg report indicates it is a deeper shift of philosophy rather than a one-time focus for iOS 12 followed by a return to the status quo.<\/p>\n<\/blockquote>\n\n

Nick Heer<\/a>:<\/p>\n

\n

If the changes are as modest as this report makes them out to be, how much of an improvement can we realistically expect in software quality?<\/p>\n<\/blockquote>\n\n

Tim Bradshaw<\/a>:<\/p>\n

\n

First Apple shareholder question is about software quality, which he says is “very unsatisfactory”. “We are getting plenty of changes but not many improvements... My solution has been to stop upgrading because I no longer trust Apple.” Apple is “losing touch with working people”<\/p>\n<\/blockquote>\n\n

Update (2018-02-14): Riccardo Mori<\/a>:<\/p>\n

\n

While I’m certain there are still underlying issues left unsolved in both Tiger and Leopard, in day-to-day general use, nothing prominent shows up on my radar. I turn on this PowerBook, it boots into Mac OS X 10.5.8, I open whatever apps I need for this session, and I feel I’m working in a stable, predictable environment. The only unfortunate thing I notice is that in places the hardware shows its age, or that certain features or services are too new to support this platform, but neither this particular vintage Mac nor its Mac OS X version are at fault. And it’s pretty amazing I’m still being productive with a 14-year old machine.<\/p>\n

[…]<\/p>\n

I’m just an outside observer, with perhaps the vantage point of having been using Apple hardware for almost 30 years. I can’t say with certainty that today both Mac OS and iOS have more bugs and issues than before. I’m also not saying that everything was 100% perfect before and now it’s all rubbish, because it’s not true. But from having extensively used (almost) each version of Mac OS and iOS, what I do<\/em> notice is that behind the scenes there was a different approach to their development before a certain point in Mac OS X’s timeline, and that something changed (for the worse) after that point.<\/p>\n<\/blockquote>\n\n

Update (2018-02-16): See also: Download<\/a>.<\/p>\n\n

Update (2018-02-21): See also: The Talk Show<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"

Mark Gurman (tweet, MacRumors, Reddit, Hacker News, ArsTechnica): Instead of keeping engineers on a relentless annual schedule and cramming features into a single update, Apple will start focusing on the next two years of updates for its iPhone and iPad operating system, according to people familiar with the change. The company will continue to update […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"2021-09-06T14:20:54Z","apple_news_api_id":"c6316812-36f1-4f78-888a-6c42bad164a2","apple_news_api_modified_at":"2021-09-06T14:20:55Z","apple_news_api_revision":"AAAAAAAAAAD\/\/\/\/\/\/\/\/\/\/w==","apple_news_api_share_url":"https:\/\/apple.news\/AxjFoEjbxT3iIimxCutFkog","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[2],"tags":[38,1143,44,31,1610,30,1609,2109],"class_list":["post-20483","post","type-post","status-publish","format-standard","hentry","category-technology","tag-apple","tag-apple-software-quality","tag-bugtracking","tag-ios","tag-ios-12","tag-mac","tag-macos-10-14","tag-radar-and-feedback-assistant"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/20483","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=20483"}],"version-history":[{"count":13,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/20483\/revisions"}],"predecessor-version":[{"id":20650,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/20483\/revisions\/20650"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=20483"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=20483"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=20483"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}