Incentives in Product Design and Development
The Twitter change where they switched everyone from their choice of timeline to ranked timeline reminds me of a question: why don’t social media companies react to negative signals from users?
The change trained me to stop checking the app since it filled my feed with garbage.
FB did the same thing quite a while ago and I asked an FB PM why, despite clicking “hide story” for months on all of the garbage in my feed, my feed was still full of high-engagement garbage instead of the lower engagement content that I’d actually want to see.
His answer was that personalization signals have low impact; something that’s 100x as likely to get engagement should completely dominate the ranking algorithm.
[…]
Until Tik Tok, no major social media app that I know of actually took user preferences seriously, so PMs could semi-plausibly say that ignoring user preferences is the right call, but Tik Tok actually adjusts to what users want and it was huge growth advantage for them.
[…]
When MS rolled out Windows 10, the plan was to force reboot & update machines with no notice.
Many engineers objected that rebooting machines with no notice was problematic, but they were overruled by PMs who said “engineers aren’t normal users, you don’t know what users want”.
[…]
Lest anyone think I’m picking on MS, Google is actually much worse about this than MS.
An eye-opening thing working at a tech company with a consumer focus (eg Google, Meta, Spotify): seeing how utterly broken customer complaints handling at scale is.
[…]
When you do try to help with an inbound smile this, you realize how the focus on revenue and impact disincentivises fixing one-off issues.
Most tickets you open will be closed by teams owning the issue as “cannot repro”, “on the backlog already w low priority” or stay idle.
And the irony is that two years into working at the company, you also “get it”.
[…]
Very few if any people and team get rewarded for fixing low impact issues impacting 0.002% of users. Also few get punished by not doing so.
[…]
To sum it up: fixing individual issues doesn’t scale, and this is why large companies focused on profits don’t do it.
Via Michael Love:
And this one of the many downsides for small businesses in relying on a bigger business (particularly one you can’t easily switch away from) for a critical part of your operations.
More generally, in many markets, consumers are uninformed and it’s fairly difficult to figure out which products are even half decent, let alone good. When people happen to choose a product or service that’s right for them, it’s often for the wrong reasons. For example, in my social circles, there have been two waves of people migrating from iPhones to Android phones over the past few years. Both waves happened due to Apple PR snafus which caused a lot of people to think that iPhones were terrible at something when, in fact, they were better at that thing than Android phones. Luckily, iPhones aren’t strictly superior to Android phones and many people who switched got a device that was better for them because they were previously using an iPhone due to good Apple PR, causing their errors to cancel out. But, when people are mostly making decisions off of marketing and PR and don’t have access to good information, there’s no particular reason to think that a product being generally better or even strictly superior will result in that winning and the worse product losing.
[…]
So far, we’ve looked at the difficulty of getting the right product or service at a personal level, but this problem also exists at the firm level and is often worse because the markets tend to be thinner, with fewer products available as well as opaque, "call us" pricing. Some commonly repeated advice is that firms should focus on their "core competencies" and outsource everything else (e.g., Joel Spolsky, Gene Kim, Will Larson, Camille Fournier, etc., all say this), but if we look mid-sized tech companies, we can see that they often need to have in-house expertise that’s far outside what anyone would consider their core competency unless, e.g., every social media company has kernel expertise as a core competency. In principle, firms can outsource this kind of work, but people I know who’ve relied on outsourcing, e.g., kernel expertise to consultants or application engineers on a support contract, have been very unhappy with the results compared to what they can get by hiring dedicated engineers, both in absolute terms (support frequently doesn’t come up with a satisfactory resolution in weeks or months, even when it’s one a good engineer could solve in days) and for the money (despite engineers being expensive, large support contracts can often cost more than an engineer while delivering worse service than an engineer).
[…]
The pervasiveness of decisions like the above, technical decisions made without serious technical consideration, is a major reason that the selection pressure on companies to make good products is so weak. There is some pressure, but it’s noisy enough that successful companies often route around making a product that works, like in the Mongo example from above, where Mongo’s decision to loudly repeat demonstrably bogus performance claims and making demonstrably false correctness claims was, from a business standpoint, superior to focusing on actual correctness and performance; by focusing their resources where it mattered for the business, they managed to outcompete companies that made the mistake of devoting serious resources to performance and correctness.
It turns out the same thing can happen in technology companies that get monopolies, like IBM or Xerox. If you were a product person at IBM or Xerox, so you make a better copier or computer. So what? When you have monopoly market share, the company’s not any more successful.
So the people that can make the company more successful are sales and marketing people, and they end up running the companies. And the product people get driven out of the decision making forums, and the companies forget what it means to make great products. The product sensibility and the product genius that brought them to that monopolistic position gets rotted out by people running these companies that have no conception of a good product versus a bad product.
They have no conception of the craftsmanship that’s required to take a good idea and turn it into a good product. And they really have no feeling in their hearts, usually, about wanting to really help the customers.
Previously:
- Instagram Reintroduces Chronological Feed Option
- “Link in Bio” Is Apparently a Billion-Dollar Business
- Twitter Makes It Harder to Choose Reverse Chronological Timeline
- Apple Music Feedback Made It to Cook
- Why Little Bugs Need to Get Fixed
Update (2022-04-12): Arvind Narayanan:
My university just announced that it’s dumping Blackboard, and there was much rejoicing. Why is Blackboard universally reviled? There’s a standard story of why “enterprise software” sucks. If you’ll bear with me, I think this is best appreciated by talking about… baby clothes!
Previously:
Update (2022-06-09): Nick Lockwood:
We’ve all heard of the 80-20 rule right? That the last 20% of development takes 80% of the effort?
The reality is actually worse. Development is like Zeno’s paradox, where effort rises exponentially.
[…]
At any point in a product’s lifecycle, it will be the same (or more) effort to fix a small bug or papercut than to implement a major new feature (or rather, to implement 80% of it, minus the last few bugs and papercuts).
Update (2024-12-05): David Chisnall:
My experience at Microsoft was that the culture revolves around lying to management.
The entire incentive system (bonuses and promotions) revolves around being able to demonstrate short-term impact. Claiming that a thing is done and ready when it’s actually an early prototype is rewarded.
[…]
If you’re managing a team, your best strategy is to over claim on the status and make sure that meetings with PMs and your management exclude any engineers who might tell the truth (and you promote the ones who are willing to lie, because that helps you).
Previously:
4 Comments RSS · Twitter
It's sad that the same prioritisation of marketing over product happened to Apple, too.
Contrast how this boardgame designer ticks about low impact issues
tl;dr he uses the percentage to calculate how many actual people it will effect and acts from there
https://www.getrevue.co/profile/gengelstein/issues/gametek-14-small-chances-big-problems-1053991
In the old days, there was no internet. Only computer magazines. They were small affairs. The guys who wrote for them were geeks, and the reviews were often pretty brutally honest. English majors weren't interested in such unimportant things as computers, so they stayed well away, writing poetry or "important narratives". Software was considered expensive, and proper reviews mattered.
Now, we have the internet. Tech is considered to be important because it's attached to money. "Review shops" want to be paid (a lot) to feature your app. Few people read magazines, and the level of their content is abysmal, because most of their "tech" writers are pretty ignorant tech-wise. Apps are cheap and far too plentiful. Making money seems to depend on selling adverts. So instead of being a business limited by the quality of engineers, software has become a volume business mostly limited by capital.
It's not surprising that this leads to dysfunctional outcomes. But it's not just software. Appliances such as furnaces don't last very long, on purpose. I wonder whether the coming recession/depression or in the worst case world war will change people's tolerance for shoddy quality.
"Review shops" want to be paid (a lot) to feature your app. Few people read magazines, and the level of their content is abysmal, because most of their "tech" writers are pretty ignorant tech-wise."
I think what you're saying might be true for some types of software reviews (I don't think there's a general audience media outlet that reviews Twitter updates), but I just don't see how it is true in general. Tech media has never been better than it is now. Instead of the partisan computer magazines of yore, we now have non-partisan, highly technical media outlets that do the kind of extensive testing no magazine has ever paid for in the past. In the history of computing, there has never been better PC case testing than what Gamers Nexus is doing right now, for example.
