Archive for November 29, 2017

Wednesday, November 29, 2017

Swift and GPU Compiler Details

Implementing Swift Generics:

Swift is a safe and efficient systems language with support for generic programming via its static type system. Various existing implementations of generic programming use either a uniform runtime representation for values (e.g., Java generics) or compile-time monomorphization (e.g., C++, Rust). Swift takes a “dictionary-passing” approach, similar to type-classes in Haskell, using reified type metadata to allow generic code to abstract over the memory layout of data types and avoid boxing. In this talk, we will describe the compilation of Swift’s generics from the type checker down to LLVM IR lowering and interaction with the Swift runtime, illustrating the how the core representation of generics flows through the system, from answering type-checking queries to the calling convention of generic functions and runtime representation of the “dictionaries”.

Slava Pestov:

If you’ve seen our Swift generics talk, this document will shed some light on generic signature canonicalization.

See also: Marcin Krzyzanowski’s Slow Swift talk.

Apple LLVM GPU Compiler: Embedded Dragons:

The adoption of LLVM to develop GPU compilers has been increasing substantially over the years, thanks to the flexibility of the LLVM framework. At Apple, we build LLVM-based GPU compilers to serve the embedded GPUs in all our products.The GPU compiler stack is fully LLVM based. In this talk, we will provide an overview of how we leverage LLVM to implement our GPU compiler: in particular we will provide details about the pipeline we use and we will describe some of the custom passes we added to the LLVM framework that we are considering to contribute to the community. Additionally, we will discuss some of the challenges we face in building a fast GPU compiler that generates performant code.

Why Little Bugs Need to Get Fixed

Joe Rossignol:

When affected users type the word “it” into a text field, the keyboard first shows “I.T” as a QuickType suggestion. After tapping the space key, the word “it” automatically changes to “I.T” without actually tapping the predictive suggestion.

Via Nick Heer (tweet):

It’s alarming to see a recurring theme of bugs in Apple’s software and hardware input devices. From dust under MacBook Pro keyboards to this autocorrect bug and the other autocorrect bug, it’s a worrying sign. Then there’s the noticeable lag when using a Magic Trackpad 2 in El Capitan or later, and the seemingly-random capitalization of words on iOS.

I don’t know how accurate the broken windows theory is, nor how appropriate it would necessarily be to compare it to problems with input devices. But it kind of feels as though the occasional usability irritants — interactivity-blocking animations, occasional layout bugs, and the like — have been ignored as a cost of a rapid development cycle. It seems like the tolerance of these kinds of bugs has built up to the point where input device bugs are now shipping.

One issue is Apple shipping bugs that should have been caught. To a certain extent, you can just chalk this up to people making mistakes, as humans do. You could perhaps blame the rigid schedule and the number of new features Apple decided to put into that major release.

But what about the little bugs that hang around for multiple major releases? Those are evidence of a process that doesn’t value quality. If Apple can’t fix bugs faster than it creates them, the only possible outcome is operating systems and apps that get buggier and buggier. This is a vicious cycle that is demoralizing for customers, and especially for the people who send in bug reports for free. If Apple can’t pay off this technical debt in a time of record earnings, stock price, and expansion, when can it?

Nick Heer:

Maybe I’m being too harsh lately with all my harping on bugs. But it’s about trust and value. I trust that I can use this software and hardware to do my job, and I paid decent money for it, so it would be nice if it were less broken.

Steve Uffelman:

I’ve seen more problems with iOS 11 and macOS High Sierra than with any other Apple releases in recent memory.

Tanner Bennett :

I’ve created a Moment full of iOS 11 and High Sierra bugs.

Steve Randy Waldman:

apple has so much money. it is constantly sending money back to shareholders. while its computer and computer software businesses are left to wither, their products increasingly shoddy, breaking the hearts of customers who stuck with the firm for decades. cool watch bands tho.

Maynard Handley:

Display multiple windows without crashing the display in bizarre ways? Not drop BT trackpad connection? Not crash on a hard disk error?

Let’s get the bugs fixed, then we can think about features. Same damn complaint for the past three years...

Even something essential like Spotlight is so polluted by bugs (dift each release but always present) and lousy UI in the face of dusk spin up, that it’s depressing to use and one is scared to suggest new features.

We don’t want engineers to feel bad, we want process to be fixed. There is adequate process for capturing and handling crashes and memory leaks, terrible process for broken UI, utterly hostile process for broken design.

And that is why they are angry when you deny that there is a problem, or say that “Apple feels your pain”. We don’t want happy words, we want the reversal of what are clearly deliberate decisions to simply stop caring about large areas of functionality.

On the mac side, how, for the love of god, is this acceptable? Happens on an iMac, no monitor plugged in, at least three times a week. Apparently randomly, and only solution is to reboot.

Apple used to care about details. And much of the company still does, but not all. A lot of crap is being shipped by people who don’t care about details, and their managers don’t care enough to notice, or to straighten them out.

There’s a LOT of this sort of crap that, as I say, is not captured by Apple’s automatic logging/tracking infrastructure; it just manifests as people rebooting and cursing --- and losing their love for Apple.

Another issue is that Apple has no PROCESS in place for handling complaints that are not traditional bugs. They track (mostly automatically, mostly successfully) crashes, hangs, memory leaks. But they don’t automatically track UI bugs, or have sentiment analysis around design.

Well, is it acceptable that my non-techie friend has to reboot her MacBook once a week bcs the audio has mysteriously gone very silent? That might not lose Apple a sale, but it does lose Apple an evangelist who would never think of buying alternatives.

Previously: Low-Hanging Fruit, iOS 11 Autocorrect Bug.

Update (2017-11-29): Luc Vandal:

What are yearly macOS updates any good for if all they bring are bugs and annoyances? How about fixing external monitor support so I don’t have to force shutdown my Mac almost every time?

I find my MBP has restarted every other day because of some mysterious Kernel Panic. Forget bells and whistles, I want reliable and stable!

Clark Goble:

iOS11 really is the buggiest I’ve encountered. Visual voicemail stopped working for me and AT&T told me it was a common problem. I had to do a clean reinstall to get it to work.

There are always bugs - especially ones that affect battery life. But for basic things just not working having bluetooth and phone app be so bad for so many people is pretty surprising. Further both have a big effect on people.

Will Cosgrove:

It’s all about the forced yearly updates. No time to pay off debt. Echoed by employees I know there as well.

Evgeny Cherpak:

Apple isn’t doomed - but people starting to feel that they paying premium prices for sub premium products… wonder how long that can last on Apple brand only. Time to wake up @tim_cook and recognize you have a problem.

James Bulman:

Apple needs standing teams of software devs who are permanently associated with a particular product. Continually pulling devs off one project onto another is what is causing these persistent bugs / product stagnation.

Richard Coppola:

Apple remains a “Functionally” structured company. Unprecedented for their size. This may be taking its toll.

Update (2017-11-30): Ryan Jones:

My Apple software is more buggy than ever. I’ll be chronicling bugs in this thread and with #bugs.

Cédric Luthi:

High Sierra is a disaster, and I’m not even talking about #IAmRoot but about what happened after I installed the 10.13.1 update.


Also the issue of GPUs in 2016/2017 MBPs being stuck throttled to about 30% of normal performance after standby still isn’t fixed, driving me and other people nuts. Restarting has become a daily routine. No response to radar to this day.

Jeff Johnson:

What’s changed is this:

Snow Leopard had 2 full years of bug fixes. Since Lion, Apple has released major Mac updates every year, mostly on a 12 month schedule. Introducing bugs faster than they can fix.

Howard Oakley:

While we’re all thinking about Apple’s software quality assurance, following its recent root user vulnerability, I’d like a few words about Disk Utility.

Peter Steinberger:

IAP purchases on macOS are broken when using Touch ID. Stable doesn’t work, 10.13.2b5 also doesn‘t fix it.

I guess nobody using Mac App Store apps anymore?

Update (2017-12-01): Lloyd Chambers:

Curiously, the configuration dialog does show icons with labels, but when dragged into the toolbar, the labels disappear. It shows a inattention to detail: if the icons need labels in the configuration dialog, why not in the toolbar?

On my Mac, Mail lets me show the toolbar labels in the main window but not in the message windows.

Jeff Johnson:

It’s little things like wonky smart folders in Mail app. They used to work perfectly, now they randomly forget emails.

Matt Long:

This interface in Xcode 9 is more evidence Apple is no longer dog-fooding. What is going on over there?

Nicholas Riley:

Family member called to report a Family Sharing app refused to launch claiming it was no longer shared. App Store wanted to charge her.

But the app clearly claims in the App Store that it’s supported by Family Sharing. What the…?

Update (2017-12-02): Jesse Squires:

Good round up of software quality problems at Apple. Although, we’ve been saying this for years now and nothing has changed. 😭

Maynard Handley:

And if you use multiple macs with screen-sharing, after a few days remote screens will no longer capture command-space and command-tab keystrokes…

(On the plus side, now that I have to reboot my Mac 4x+ a week, this is a problem encountered less frequently...)

Mark Munz:

1. I used to have months & months w/o system-wide crash.

2. I got a new iMac + High Sierra.

3. Now I wake up to a crashed Mac EVERY SINGLE MORNING!

Update (2017-12-05): See also: Accidental Tech Podcast.

Ilja A. Iwas:

For some of our users running 10.13, certain values stored in NSUserDefaults are lost upon app restart. Anybody else seeing this?

Steve Troughton-Smith:

iPad Pro 12.9" Smart Keyboard owners: has your Smart Keyboard suddenly become incredibly unreliable since the release of iOS 11 in September?

Antonio Mikatović:

Hello, is anyone aware of the bug where flash on iPhone 8Plus and iPhone X doesn’t work in cold weather? Flashlight works fine, flash for photos does not. Im having the problem on my X.

Samer Farha:

It’s not just iOS or macOS, either. tvOS is pretty much unusable. The Computer app requires a force quit for every other show watched.

John Gordon:

Contact search still broken in 11.2. Returns empty Contact.

Update (2017-12-06): Steven Frank:

Why can’t computers wake from sleep reliably?

Like imagine spending $2-3,000 on literally anything and it doesn’t always turn on/off properly and going oh, yeah, it just does that sometimes and everyone being fine with that.

Ryan Jones:

As usual, my MBP is completely dead after going to sleep with full battery.

Update (2017-12-07): Marco Arment:

Disabling font smoothing is STILL broken in High Sierra 10.13.2[…]

Maynard Handley:

Well that didn’t take long!

Installed 10.13.2 at around 10:am.

Screen corruption by 7:00pm.

Hell of an OS you have there, Apple!

HTF is a broken graphics stack not the highest bug fix priority?

And, after a brief month or so of iOS-macOS WiFi sync actually WORKING, we’re back to it completely broken. Just like in Sierra and El Capitan.

After it used to work flawlessly in Yosemite.

James Thomson:

Sigh, 10.13.2 doesn’t fix the random black frames in the PCalc About screen on Intel built-in video cards, if anything it’s worse…

Update (2017-12-08): Jason Snell:

My hope is that these missteps lead to an analysis of Apple’s internal processes that leads to changes that improve the quality of Apple’s software. I believe that Apple can effect that change if it wants to.

Nick Heer:

This thing where the mouse cursor becomes unresponsive during heavy network activity has been a bug since Sierra. It’s probably worth fixing.

Update (2017-12-13): Steven Woolgar:

I’ve had my issues with macOS releases (cough Lion cough), but High Sierra is far away the most buggy macOS I’ve used to date (and I’ve used them all). I didn’t even install it until 10.13.2! I feel the same way about the latest iOS version. 🤞🏽 to improvements.

Update (2017-12-13): Marco Arment:

Damn, another High Sierra point release likely to pass without fixing font smoothing.

Settings, General, “Use LCD font smoothing when available”

It’s subpixel antialiasing, making fonts look sharper on low-res displays but thick and blurry on Retina (in my opinion).

In High Sierra, with it disabled, text truncated with an ellipsis renders in AA mode anyway.

Marco Arment:

Holding onto Sierra on my iMac is becoming untenable.

My iCloud photos are all in HEIF, I can’t receive AirDrop from my phone anymore, and now, iMessages from my phone are starting not to show up on the iMac.

I think it’s over.

Update (2017-12-15): Alex J Burke:

Honestly, High Sierra has seriously damaged the Mac for me. I regret the upgrade - none of us with Touch Bar MBPs can plug into external monitors without flickering, I’ve had weird beach balls that require reboot, on and on. Never experienced this in 11 years of OS X.

Harald Wagener:

After a year of absence on the macOS platform, I have recently returned. And I see all issues that I experienced on Linux and some: Weird unlock bugs (chrome windows on top of screen unlock), wake-from-sleep bugs (suddenly showing error below), seconday monitor glitches, phantom touchpad events, weird wifi behavior, ... my four year old chromebook pixel behaves better than this. @googlechrome if you'd offer the Pixelbook in Germany, I'd buy one yesterday.

Update (2017-12-20): Adrian Kingsley-Hughes (via Slashdot):

Well, Steve is now long gone, and so it the ethos of “it just works.”

Update (2017-12-28): Scott Alexander:

Dear @apple - your OS has a global spellcheck that autocorrects names of medications to names of different medications, eg “duloxetine” to “fluoxetine”, without telling the user. Some clinics inexplicably continue to use MacBooks. Please fix this before someone gets hurt.

Update (2018-01-03): Nicholas Riley:

A list of macOS, tvOS and iOS bugs I encountered while helping my family over the holidays

Update (2018-01-05): Khoi Vinh:

iOS 11 tip: My 9.7-in iPad Pro slowed down significantly when I upgraded to iOS 11. During the holiday break I wiped it clean and reinstalled everything from scratch—not from a backup but app by app. It now runs MUCH faster. It was a huge pain to do but well worth the trouble.

Update (2018-01-15): Steven Frank:

A Mac I set up from scratch about 2 days ago is already kernel panicking in sleep and I just, I am just so tired of the entire thing

Colin Cornaby:

Literally every Mac I have does this in High Sierra and has since the developer betas.

Reliable sleep used to be a key feature. I have no idea what is going on with Apple.

Ryan Jones:

Is copy/paste between Mac and iOS one of those things we all know doesn’t work? Or just not for me.

Update (2018-02-19): Geoff Duncan:

Dear Apple: Want to know why Mac users dread updating software? Because this happens about 50% of the time.

Update (2018-06-03): Geoff Duncan:

I think Apple Contacts is on a mission to humiliate me. It has lost track of someone I’ve had in Contacts since literally my first day using it, and resurrected contacts I deleted six years ago.

High Sierra Bug Allows Root Access With Blank Password

chethan177, in the Apple Developer Forums, two weeks ago (via Mike Myers):

If you’re unable to login at startup using username: root and empty password, then login with your existing account (standard user).

Again, head over to System Preferences>Users & Groups. Click on the Lock Icon. When prompted for username and password, type username: root and leave the password empty. Press enter. This might throw an error, but try again immediately with the same username: root and empty password. This should unlock the Lock Icon.

@jeremydmiller78, a week ago, posted a video.

Lemi Orhan Ergin, yesterday (Hacker News):

Dear @AppleSupport, we noticed a *HUGE* security issue at MacOS High Sierra. Anyone can login as “root” with empty password after clicking on login button several times. Are you aware of it @Apple?

Juli Clover:

There appears to be a serious bug in macOS High Sierra that enables the root superuser on a Mac with a blank password and no security check.

Adam C. Engst:

Wait, it gets worse. I’ve confirmed that if you have Screen Sharing (or Remote Management) enabled in System Preferences > Sharing, someone can connect to your Mac over the local network or, depending on your Internet setup, the outside world. I did this from a guest account on my MacBook Air and ended up at a login window on my iMac, from which I was able to click the Other button, enter root and no password in the appropriate fields, and create a root user account on my iMac.

The practical upshot is that anyone who has local or network access to your Mac can log in and access all files with impunity.

Rene Ritchie:

“We are working on a software update to address this issue,” an Apple spokesperson told iMore. “In the meantime, setting a root password prevents unauthorized access to your Mac. To enable the Root User and set a password, please follow the instructions here. If a Root User is already enabled, to ensure a blank password is not set, please follow the instructions from the ‘Change the root password’ section.”

Juli Clover:

Disabling the root user account again follows the same steps, but at the “Edit” portion of the process, you’ll select “Disable Root User” to remove the option. Until the bug is fixed, though, you’ll want to leave the root user account intact to prevent it from being accessed without a password.

Ilja A. Iwas:

Not impressed by Apple’s poor handling of yesterday’s 0-day. No e-mail, no mention on, only a KB-link circulated in the media that doesn’t even acknowledge the issue.

John Gruber:

I rarely describe any bug as inexcusable, but this is inexcusable.

Peter Maurer:

Oh how I’d love to know how they ended up with code that creates root as a side effect. “Account doesn’t exist” => “let’s create it” seems like a weird train of thought. For testing, perhaps?

Rui Carmo:

The scheduled release approach (whereby software is shipped in lockstep with increasingly predictable hardware launches) has been steadily eroding quality across the board (and iOS 11.0 was a great example of that), but macOS seems to be falling into full-fledged neglect, and as a primarily UNIX user, I’m flabbergasted this kind of thing is even possible in 2017.

Nick Heer:

I’m not deluded enough to think that complex software can ever be entirely bug-free, but I’d love to see more emphasis put on getting Apple’s updates refined next year, rather than necessarily getting them released by mid-September.


For extra irony, recall that High Sierra was pitched as a refinement of MacOS Sierra.

Previously: Encrypted APFS Volume’s Password Exposed as Hint.

Update (2017-11-29): Patrick Wardle:

Starting with the odm_RecordVerifyPassword function, it invokes an unnamed method, ‘sub_826b’. This subroutine first invokes another helper function, ‘sub_826b’, to “read shadowhash data from” from the account that the user (or attacker) is trying to log in to. For enabled accounts (such as the user account) this read will succeed as this data exists.


For disabled accounts, (such as root account that is being targeted), this information is not present, so this function will fail!


Since a non-zero value was returned, execution continues with a call to various methods such as sub_13d00. As the debug log statments in the decompilation show, these will perform an upgrade from a crypt password to a shadowhash or securetoken[…]


However, if we look at what these ‘upgrade’ subroutines are called with, it’s with the password we provided[…]

Apple has fixed the bug:

A logic error existed in the validation of credentials. This was addressed with improved credential validation.

Rene Ritchie:

Apple sent me the following statement:

When our security engineers became aware of the issue Tuesday afternoon, we immediately began working on an update that closes the security hole. This morning, as of 8:00 a.m., the update is available for download, and starting later today it will be automatically installed on all systems running the latest version (10.13.1) of macOS High Sierra.

We greatly regret this error and we apologize to all Mac users, both for releasing with this vulnerability and for the concern it has caused. Our customers deserve better. We are auditing our development processes to help prevent this from happening again.

Patrick Wardle:

Apple’s patch for #iamroot bug “improves cred validation” ... meaning they perform extra checking on the call to od_verify_crypt_password() In prev. blog posting we surmised “it should fail” - it did; appears they just didn’t check💥😅 👋🏽

Unfortunately, the update breaks file sharing.


If you experience issues with authenticating or connecting to file shares on your Mac after you install Security Update 2017-001 for macOS High Sierra 10.13.1, follow these steps to repair file sharing[…]

John Gruber:

It’s natural to speculate how a bug as egregious as the now-fixed High Sierra root login bug could escape notice for so long. It seems to have been there ever since High Sierra 10.3.0 shipped on September 25, and may have existed in the betas through the summer.

Steve Troughton-Smith:

Bad actors could have known about this since June and we’d never know, as [remote] root access to a machine would let you easily cover your tracks.

Update (2017-11-30): Jeff Johnson shows that Apple is now distributing a new version of the update, which doesn’t break file sharing. The changes seem to indicate that the file sharing problem was caused by a flaw in the updater itself, rather than in the code that was patched to fix the root issue.

Update (2017-12-01): Andy Greenberg (via Hacker News):

Those who had not yet upgraded their operating system from the original version of High Sierra, 10.13.0, to the most recent version, 10.13.1, but had downloaded the patch, say the “root” bug reappears when they install the most recent macOS system update. And worse, two of those Mac users say they’ve also tried re-installing Apple’s security patch after that upgrade, only to find that the “root” problem still persists until they reboot their computer, with no warning that a reboot is necessary.

Update (2017-12-05): Jeff Johnson:

AFAICT you don’t get the so-called “automatic” security update 2017-001 as long as you have all the prefs off?

Howard Oakley:

These are indicators that Apple is in the process of rewriting a lot of macOS. In the case of the root user vulnerability, this was in Open Directory, whose source code may well date back ten years or more. It has been my contention that macOS-only code, such as Time Machine, gets considerably less resources than that which is shared with iOS, but that doesn’t explain why, if such macOS development is being poorly resourced, Apple’s precious engineers are busy rewriting systems such as Open Directory, when there are so many other demands.

Thomas Reed:

Since the update doesn’t require a restart, and since many Mac users can be rather averse to restarting, this means that people upgrading from 10.13.0 to 10.13.1 could easily end up being vulnerable to this bug for weeks or months, until they next decide to restart. Keep in mind that nearly all 10.13.0 users have probably already had Security Update 2017-001 installed automatically at this point, putting them into a pipeline heading straight for this issue.

Update (2017-12-11): Accidental Tech Podcast says that the bug was actually reported to Apple via the proper channels around the same time as chethan177’s forum post.

Network Neutrality, Ajit Pai, and Title II

Tim Wu:

On Tuesday, the F.C.C. chairman, Ajit Pai, announced plans to eliminate even the most basic net neutrality protections — including the ban on blocking — replacing them with a “transparency” regime enforced by the Federal Trade Commission. “Transparency,” of course, is a euphemism for “doing nothing.” Companies like Madison River, it seems, will soon be able to block internet calls so long as they disclose the blocking (presumably in fine print). Indeed, a broadband carrier like AT&T, if it wanted, might even practice internet censorship akin to that of the Chinese state, blocking its critics and promoting its own agenda.


The problem for Mr. Pai is that government agencies are not free to abruptly reverse longstanding rules on which many have relied without a good reason, such as a change in factual circumstances. A mere change in F.C.C. ideology isn’t enough. As the Supreme Court has said, a federal agency must “examine the relevant data and articulate a satisfactory explanation for its action.” Given that net neutrality rules have been a huge success by most measures, the justification for killing them would have to be very strong.

The idea of network neutrality is very appealing, but I find this editorial by the coiner of that term rather unconvincing. His main example is the Madison River case, where Vonage was fined for violating antitrust rules. This was 10 years before network neutrality became law. Where is the evidence of the “huge success” as a result of the 2015 change? Most of the examples that people cite seem to not actually be cases where network neutrality was relevant. In fairness, the evidence that the law has caused harm also seems to be weak.

We need to weigh the potential costs and benefits of keeping vs. repealing Title II and also look at what would happen if we’re wrong. It seems to me that, if it’s repealed and then there is abuse, this would be a relatively easy thing for Congress or the FCC to address. There would be a clear goal with a lot of popular support. Whereas, it is difficult to see which opportunities and innovations Title II might be preventing, and so there is no mechanism for a possible course correction from that side. So I think the structure of the issue favors a wait-and-see approach.

Via Tyler Cowen:

Keep in mind, I’ve favored net neutrality for most of my history as a blogger.


If you are wondering why I have changed my mind, it is a mix of new evidence coming in, experience over the 2014-present period, relative assessment of the arguments on each side moving against NN proponents, and the natural logic of the embedded trade-offs, whereby net neutrality typically works in a short enough short run but over enough time more pricing is needed. Of course it is a judgment call as to when the extra pricing should kick in.

Elizabeth Harrington:

Pai said it is necessary to repeal the net neutrality rules because of their effect on broadband investment. Capital expenditure in broadband declined by 5.6 percent since Title II was adopted in 2015, which amounted to over $3.6 billion in lost investment.

“These heavy-handed regulations as we find in the order are having an effect on investment and innovation, making companies less likely to raise and spend capital building out networks, especially in rural, and low-income urban America,” he said.

I don’t find this very convincing, either. I don’t see how the repeal would incentivize Comcast to upgrade my local infrastructure or make it possible for another ISP to compete with them. Frankly, I don’t think the fight is about helping those of us in rural areas with little competition. It’s primarily a struggle between two different groups of large companies, the ISPs and carriers vs. the tech giants who fill their bandwidth.

Ben Thompson (Hacker News):

Of course ISPs should be neutral — again, who could be against such a thing? What is missing in the ongoing debate, though, is the recognition that, ever since the demise of AOL, they have been. The FCC’s 2015 approach to net neutrality is solving problems as fake as the image in Wu’s tweet; unfortunately the costs are just as real as those in Congressman Khanna’s tweet, but massively more expensive.


To put it another way, given the stakes, the benefit from regulation must be massive, which is why the “net neutrality” framing is so powerful: I’ll say it again — who can be against net neutrality? Telling stories about speech being restricted or new companies being unable to pay to access customers tap into both the Internet’s clear impact and the foregone opportunity cost I just described — businesses that are never built.

That, though, is exactly the problem: opportunity costs are a reason to not regulate; clear evidence of harm are the reasons to do so despite the costs. What is so backwards about this entire debate is that those in favor of regulation are adopting the arguments of anti-regulators — postulating about future harms and foregone opportunities — while pursuing a regulatory approach that is only justified in the face of actual harm.

The fact of the matter is there is no evidence that harm exists in the sort of systematic way that justifies heavily regulating ISPs; the evidence that does exist suggests that current regulatory structures handle bad actors perfectly well. The only future to fear is the one we never discover because we gave up on the approach that has already brought us so far.


And, I’d add, if neutrality and foreclosed competition are the issue net neutrality proponents say they are, then Google and Facebook are even bigger concerns than ISPs: both are super-aggregators with unprecedented power and the deepest moats ever seen in technology, and an increasing willingness to not be neutral.

John Gruber:

The key idea to keep in mind is that the basic principles of “net neutrality” and the specific regulations put in place by the Obama administration in 2015 are different things. You can be in favor of net neutrality in principle but be opposed to the current regulatory structure as the best way to achieve and protect it.

Geoff Duncan:

All these claims are dubious. Where the FCC says broadband investment has fallen since 2015, ISPs have consistently told their investors (via legally-binding financial disclosures) that net neutrality regulations were not impeding them. Almost every major network operator — from Comcast and Verizon to Time-Warner, Sprint, and T-Mobile — has engaged or is actively engaging in some form of blocking, paid prioritization, or (particularly) throttling with little or no disclosure to customers. And network operators didn’t set up fast lanes in the “light-touch” regulatory era before 2015 because they were waiting to see how a number of court challenges to FCC authority were going to turn out.


When proposing to undo net neutrality, Pai promised a “far more transparent” process than that used by the FCC in 2015. Yet the process the FCC implemented this time around apparently gave no thought to filtering out automated spam and trolling of the comment process, leading to the FCC claiming that it was just too burdensome on them to, you know, actually process the comments. Also too burdensome? Publishing the comments, or responding to inquiries about the comment process. In the words of Commissioner Mignon Clyburn, the process completely ignored “thousands of consumer complaints and millions of individual comments that ask the FCC to save net neutrality and uphold the principles that all traffic should be created equal.”

Nick Heer:

Recently, Verizon began throttling video streaming on their cellular network, too, with the exception of its NFL app which, by the way, is also exempt from data caps. The FCC under Tom Wheeler said that AT&T was violating net neutrality rules when they exempted their own DirecTV service from users’ data caps, too, giving it an unfair advantage over other streaming video services. Comcast hilariously argued that their broadband-powered service for streaming video to laptops was exempt from the anticompetitive agreement they signed when they acquired NBCUniversal.


There is clearly plenty of evidence that ISPs will not treat data the same if offered the opportunity to do otherwise. And, I stress again, we aren’t simply talking about internet providers here — these are vertically-integrated media conglomerates which absolutely have incentive to treat traffic from friendly entities differently through, for example, zero-rating, as AT&T did with DirecTV, Verizon does with their NFL app, and T-Mobile does for certain services.


In fact, zero-rating is, in general, covered by the 2015 net neutrality rules. That’s why the FCC sent a letters to AT&T and Verizon stating that aspects of those companies’ zero-rating practices discriminated against competitors.

Ben Thompson:

To summarize the takeaways:

  • Unregulated cable broadband grew faster than highly-regulated DSL
  • Removing the mandate that telephone companies open up their networks was correlated with a significant increase in DSL growth relative to cable, suggesting increased investment
  • Harmonizing regulation further increased DSL growth relative to cable (and, from the vantage point of 2017, precipitated significant investments in fiber offerings)

Note the Canadian broadband control set: there was not a similar shift in DSL numbers in Canada, suggesting that it is unlikely a secular technology shift drove these numbers.


The question that must be grappled with, though, is whether or not the Internet is “done.” By that I mean that today’s bandwidth is all we all never need, which means we can risk chilling investment through prophylactic regulation and the elimination of price signals that may spur infrastructure build-out (that being the elimination of paid prioritization).

Previously: Network Neutrality.

Update (2017-12-01): Jared Newman:

Absent some new legislation, the current Title II rules are the only protection consumers have against zero-rating. Although the practice has some consumer-friendly uses—T-Mobile’s Binge On program, which is open to all streaming video services at no cost, is one example—it also allows for anti-competitive behavior. Investigating the latter would be a way to keep internet providers honest.

All of which may explain how cable companies and telcos can now claim to support net neutrality, and how Comcast can specify that it’s against the notion of internet “fast lanes” and “anti-competitive paid prioritization.” Those tools are no longer necessary to gain an advantage in streaming video. The ever-present threat of data caps can do the heavy lifting instead.

Update (2017-12-07): See also: Exponent and Mike Masnick (via Nick Heer).

Update (2017-12-08): Nick Heer:

Pai has claimed that his proposed rollback will encourage net neutrality practices without regulation because it will require ISPs to be fully transparent. In a shocking turn of events for statements and policies originating from the top minds of this administration, that claim turns out to be a complete lie: ISPs won’t have to be as open and transparent about their pricing and policies, and they have repeatedly stated that they would use tactics like paid prioritization to manipulate network traffic if given the opportunity.