Archive for September 23, 2005

Friday, September 23, 2005

Macworld Browser Review

Josh Aas (Firefox developer):

Secondly, Macworld put Firefox 1.0.6 ahead of Safari, which is almost as ridiculous as putting Camino behind Opera. See, I’m not biased towards Mozilla browsers! Safari 2.0 is simply a better browser than Firefox 1.0.6. Its [sic] not even close. The story is a little different with Firefox 1.5b1, but that isn’t what they are reviewing.

I agree that the Macworld rankings are odd. I think I would order them Safari, OmniWeb, Camino, Firefox, Opera. I couldn’t care less about Firefox themes or extensions, and I deduct huge numbers of points from it and Opera because they don’t look or feel like Mac apps.

Unit Testing

Wil Shipley’s heart is in the right place. It’s obvious that he cares about quality, and his experience and common sense rightly make him skeptical of process. But I think he’s absolutely wrong about unit testing. In brief, I think he misunderstands what unit testing is (or should be) and what it’s useful for. He shoots down a stupid way of doing testing, throws the baby out with the bathwater, and concludes that it’s better to write good code in the first place, do manual testing (“Try odd things. Whack keys.”), and rely on beta testers.

I don’t deny that this approach has worked for him. He’s written a lot of code, people have generally been happy with the products, and they’re not known for being buggy. But my hunch is that Wil’s teams have gotten good results because they’re unusually smart, experienced, and hard-working. They’ve succeeded in spite of their testing philosophy. Wil makes a great case that manual testing and defensive programming are necessary, but I think most programmers would be more effective and efficient if they combined these with extensive automated unit testing.

Now, I’ll be the first to say that by-the-book XP/TDD goes overboard. If you need a new class or method, I think it’s a waste of time to write a test that simply checks for the existence of the class or method, then write a stub, then run the tests, and then then flip back and write the real tests. Perhaps that example is just for pedagogical purposes and one isn’t meant to do that in production code.

I also think it’s a waste of time to write tests for everything. Some parts of the code are so simple that they don’t need tests, though this applies less often than you might think. Other times, it would take so much work to write a proper test that it’s better to punt and check that item manually. (I’m thinking of user interface details that you’d immediately notice when running the application, as well as more elaborate situations/interactions. In the latter case, write the test down in English and make sure that you really do it manually before shipping.) Again, though, this applies less often than you might think.

My overall point is that time is limited, so you should use it wisely. And this is why extensive unit testing is a big win. Yes, it’s not possible for your tests to cover all the pathways through the code, with all the possible inputs. And even if they could, it probably wouldn’t be a good idea to spend your time writing tests to do that. This does not mean, however, that you should reject unit testing as impractical and try to test everything manually. That’s not a good use of time, either. In brief, proper unit testing saves time and improves quality because:

A lot of people seem to think that automated testing is only for frameworks, or for tools without graphical user interfaces and interaction, or for academics, or for software that’s directly responsible for people’s lives. Not true. I think unit tests are useful in writing Mac applications, and even for testing large chunks of the user interface (i.e. checking that a table is showing the right rows in the right order, that clicking a button has the right effect, that a certain sequence of actions causes a view to refresh in the right way, etc.).

That said, your tests will not cover everything, or even close to everything. GUI software has many sources of input—all the different user interface items, in addition to the disk, the network, etc. It also has complex interactions with huge chunks of operating system and library code that you didn’t write and don’t have access to.

So, yes, you shouldn’t believe that because you have unit tests you don’t have to do manual integration tests. I doubt anyone is in danger of thinking that. Wil says:

Testing is hugely important. Much too important to trust to machines. Test your program with actual users who have actual data, and you’ll get actual results.

And I doubt anyone would disagree with that. However, testing is also too important to trust to humans. This is one reason that you need automated unit tests. The other is that it’s not all about how many bugs users will find in the binary that you ship. It also matters how long it took you to create that binary, and how easily you’ll be able to develop the next version. People who like automated testing find that it helps them write better code more quickly. I believe it’s at least as beneficial for small teams.

Don’t miss the comments from Chris Hanson, Marcel Weiher, and others.

No PowerBook Can Hold Me

Jonathan Rentzsch:

I may have to face the fact that my days of living off a PowerBook are over, perhaps for good. If so, I’ll probably go with big-honkin’ desktop machines and one small cheapie iBook for presentations and conferences. I fear the Macintel in the mail is the first step down that slippery slope.

Backup 3.0: Web-Influenced UI

Michael McCracken:

Do we really want to doom users of desktop apps to a lot of web-style scrubbing around with the mouse to discover features?…Have we just been conditioned to click and double-click around everywhere in case we can edit the text? It seems to me that kind of behavior only makes sense inside a text editing view, not just anywhere in a window.

Hopefully, Michael’s questions are rhetorical, but in case they’re not: Apple, this design sucks. Editable fields should look editable.