Thursday, February 18, 2021

Why Reporting Bugs to Apple May Harm Software Quality

Howard Oakley (tweet):

In recent years, I’ve grown concerned – as many of you have – at the increasing number of obvious bugs in release versions of macOS. In the last week or so, I’ve come across some real howlers: the Rich Text Spotlight importer which can’t import the content of RTF documents, Bluetooth status which is never up to date when you first check it, and most recently Big Sur installers and updaters which don’t work on external SSDs connected to M1 Macs.

[…]

Although Dave tested the changes he’d made, he didn’t have time to test whether the mdimporter still indexed the content of RTF files.

It doesn’t help that recent security changes have made it more difficult to test Spotlight importers.

I’ve now reached the stage where I simply don’t have time to report all these bugs, nor should I have to. Indeed, I’ve realised that in doing so, I only help perpetuate Apple’s flawed engineering practices. […] We let Apple get away with this by devoting our time to testing and documenting for Apple. […] Above all, there’s the threat. If we don’t complete Feedback, then Apple won’t know of the bug, and it won’t get fixed.

I’m frustrated, too, and I can certainly understand not having the time to report all the bugs one encounters. But I just don’t see how filing can hurt. Apple would still “get away with it,” and the bugs would be fixed even later, if at all.

If you’re an Apple engineer, please don’t take this personally. I don’t think the current system is good for you either, but at present Apple only seems to respond to public criticism.

I haven’t seen much correlation, personally. Some very public bugs go a long time with no fix. Some obscure ones that I file get fixed right away. Some serious ones get fixed 6 months or a year after I was forced to develop a workaround. Most never get fixed and are seemingly ignored.

Dave Wood:

Got a response to a radar I filed 3 years ago, about a bug in iCloud. They say they can’t look into it now because “the server logs from that date have rolled off”.

This is why I barely file bugs with Apple anymore. Just a waste of everyone’s time.

See also: How to Report Bugs to Apple So They Get Fixed.

Previously:

Update (2021-03-15): Nick Heer:

Often, I suspect, users will not attach all of the diagnostics needed for Apple’s developers to even find the bug. But I have to wonder how effective it is to be collecting so many system reports all of the time, and whether it is making a meaningful difference to the quality of software — particularly before it is shipped. I have hundreds of open bug reports, many of which are years old and associated with “more than ten” similar reports. How can any engineering team begin to triage all of this information to fix problems that have shipped?

14 Comments RSS · Twitter

I don't report bugs to Apple.

Apple should not rely on unpaid users / third party developers to test their software. If you report bugs to Apple, you're basically agreeing that they should have this expectation. I could understand helping Apple if they were a small company and only made a few bugs. But they're a 2 trillion dollar company, and they make more than a few bugs. So I'm not going to help them, unless they were to pay a bounty like for their security holes. By all means, however, report it on your blog as a public service for other developers and users. Just don't predigest it for Apple.

Look to the leadership. This is mostly, if not all, Federighi's fault. He's Apple's Senior Vice President of Software Engineering. This mess is his doing. He needs to be called out.

Don't let his charm disarm you. Ignore what he says, watch what he does. The lack of documentation problem that devs have been complaining about for years? His fault. Turning macOS into a locked down OS like iOS? His fault. Bugs, bugs, bugs that go unfixed for years. His fault.

The man is a hack. He's the software engineering equivalent of J.J. Abrams.

I read over that article about "how to report bugs to Apple". I've followed all of those steps. All of my bug reports have gone into a black hole. None of them have been fixed.

I'm through reporting bugs to Apple. It's clear they don't really care about the software quality of macOS. If they want to course correct, and good god I hope they do, it's up to them to clear out their internal dysfunction.

I believe in the mutual benefit of reporting bugs to Apple and other companies. Ideally the company ends up with better software and I get my issues resolved sooner. But it's been a long time since I felt like there was anything positive about Apple's feedback reporting process.

My last negative experience happened very recently, where I received an email from Apple asking for an update on an old bug from maybe a year ago. I didn't remember the details so I tried to view the bug in the Feedback Assistant app. The bug wasn't listed anywhere, even when I searched by the FB number. So I filed a bug against the Assistant and moved on.

But even that poor experience represents one of the better interactions I've had over the last several years. At least Apple followed-up on the issue. Mostly the problem is that you never hear anything from Apple. Or worse, they request a bunch of logs/data and then you never hear anything. As Dave Wood pointed out, the process is mostly just a waste of everybody's time. Once I was asked for a sysdiagnose in response to my enhancement request! I can't remember the last time an issue I filed was actually fixed or improved.

I don't know the solution to Apple's bug reporter problems. I sympathize that it must be an incredibly difficult firehose to manage considering Apple's scale. I have no doubt that being on the receiving side of endless low-quality bug reports is no fun for the Apple engineers either. But I think I have finally reached a point where I've decided there is literally no benefit to reporting issues, and thus I should just stop trying.

@Martin One weird thing I found is that there are now two separate lists of bugs in Feedback Assistant. One is under my name and one is under my company’s, even though there is only one developer account. I think it depends on whether I initially filed the bug from the app or the Web page. In any event, if I can’t find a bug in one list, I usually see it if I switch to the other.

Still on Mojave because no one knows if the bug that loses emails has been fixed. A basic function!

I've stopped reporting bugs as well. I did a bit with Radar, where I would get some feedback. But with the new Feedback system, its a black hole. I never get any response. Its not worth my time.

I get dealing with bug reports suck. But you gotta do it if you are a developer.

Even Radar was a black hole. Back when I participated in the cocoa-dev list in the early-mid 2000s, I drank the "file a Radar" Kool-Aid, and still have some rdar:// links in my code. Heck, I even filed doc bugs back when Apple was actively improving documentation. Unfortunately, most of my Radars were over a decade old and completely ignored by the time Apple purged them and moved to the present system, including my easy documentation bugs.

Customer Seeding got most of my bug reports, since we'd get more feedback on those than filing through Radar. That program became as useless as Radar in the 10.7 timeframe, which I guess coincides with Federighi's return to Apple. I think there are a lot of factors that led to the decline in software quality from 10.7 on:
1) no Jobs
2) Ive should never be allowed near software
3) Mac increasingly popular -> more bug reports from new Mac devs
4) Public betas instead of a small crew of fanatical volunteers (Customer Seeding)
5) Windows refugees filing bugs (how else do you explain MDI Xcode and the death of Interface Builder?)
6) Federighi is bad
7) Focus on "metrics" and sysdiagnose for bug report data mining (ok, this is speculation)

I haven't filed a bug report in years, and don't plan to start again. It's more satisfying to complain on Michael's blog, and accomplishes just as much :-).

One weird thing I found is that there are now two separate lists of bugs in Feedback Assistant. One is under my name and one is under my company’s, even though there is only one developer account.

On my company account, I now have three lists — a “Personal” one (not to be confused with the “Personal” one on my actual personal account), and one for the two teams I’m in. Perhaps if I mark one read in a team, the rest of the team also sees it as read? Otherwise, I’m not sure I see the point.

Not being able to sign in to both accounts at the same time is annoying, particularly since I often don’t know which account I’ve filed a bug under.

Last time I filed some bugs was for a beta version of Mavericks. I spent many hours narrowing down the steps to reproduce, taking screenshots (they were UI issues), writing and rewriting the descriptions to make them as clear as possible (I'm not a native speaker), etc.

I got a follow-up email about a year and a half later, in wich they were requesting I tweak some things and then try to reproduce again.

Needless to say, my machine was no longer running that beta version for quite some time, and most of the issues I had reported were already fixed in the final version anyway. I decided life's too short for that, replied "sorry" and moved on.

For a company that wants to appear detail (and customer-) oriented, some of the things they do are simply impenetrable. I'm really very sad.

@Michael Thanks for the potential tip about multiple bug lists in the Feedback Assistant, even when you only have a single developer account, but I only see one list of my submitted bugs over here.

I am not a programmer or developer, but I am sure the readers of this list are.

I have a very basic question. I have heard that when Apple wants to update software, they will put together a team of developers, that may not include any of the developers who worked on this particular app previously. Is this true?

Secondly, why don't the teams appear to communicate and collaborate among themselves? Take the new Mac Notes app. There is no preference for line spacing and the only separator for a numbered list is a "." A period. So only 1. 2. 3. and not 1) 2) 3) or 1>, 2> 3>.

But one need only go look at the free Apple App called Pages where the developers have already worked out most of these things. Why aren't the Notes team using code or designs from the Pages team?

Photos App accesses the Photos Library on Mac differently than does the Finder in Media when trying to choose a photo to upload. Finder is slow as Heinz Ketchup while the Photos App is speedy. One allows searching by location while the other does not. And there is another Photos team of engineers doing Photos for iOS, and yet another engineering team doing Photos in iCloud.

At least is there any consistency with the Product Managers?

Apple does not produce a single "best in class" piece of software, other than operating systems. Maybe Final Cut Pro but I've been out of the post-production world for quite a while.

>I have heard that when Apple wants to update software, they will put together a team of developers, that may not include any of the developers who worked on this particular app previously. Is this true?

That may be true for some apps that are rarely-updated. (Supposedly, this was true for the Disk Utility UI rewrite around High Sierra.)

I know for a fact that there are continuously-existing software teams, such as one for the iWork apps.

> Secondly, why don't the teams appear to communicate and collaborate among themselves?

The following is hearsay, but this is supposedly a culture Steve deliberately introduced when he returned in the late 1990s.

Before that, there were teams for topics like UI design which took a big-picture look, across apps, to keep things consistent. I believe the concern was that this hampered individual apps' ability to innovate; things died in design by committee, if you will.

But yes, the flipside of that is a frustrating lack of consistency. (See, for example: at one and the same event, they introduce Center Stage as a new feature on the iPad Pro. Then they introduce the iMac which doesn't have it, even though that feature would arguably make even more sense on it! Why did they omit it? I think the answer is, in part: because the two teams were entirely isolated and nobody thought or thought in time "hey, let's collaborate on that".)

I'm beginning to think submitting feedback is a waste of time too.

I haven't even been able to log in to https://beta.apple.com/ or Feedback Assistant for over six months, likely due to an issue with the Safari Developer Program. I'm told to be patient. I'm not holding my breath.

I have several hundred open reports and get email updates about comments that I cannot see. In all likelihood, the comments are probably to just try to reproduce the problem on the latest release...

Leave a Comment