Wednesday, May 11, 2022

USB-C Hubs Breaking Ethernet Networks

algalgal:

I recently learned that some USB-C hubs, with power pass through and an Ethernet port, seem to destabilize a wired network when they are connected to ethernet and to power but not to a computer. They do this by flooding the network with bad traffic (or maybe PAUSE frames).

[…]

It sounds unlikely but it seems like a longstanding issue if you look for it.

This carefully researched 2018 reddit post does the best job documenting the issue in depth, manifesting on a TOTO USB-C hub. This blog post describes the same issue with Aukey and Flyland hubs. This 2020 forum post describes the issue with an Orbi router. More recently, this 2021 forum post describes it in a Dell monitor’s embedded USB-C hub

I believe I’m seeing it myself. My new Anker PowerExpand 8-in-1’s Ethernet port lights up with a lot of “phantom” network activity even when there’s no computer attached. Then later, a computer and other devices plugged into the same network switch as that USB-C hub will lose wired connectivity. Restarting the switch resolves the problem. But the problem is also solved by never leaving the hub USB-C powered and connected without a computer!

Via Kyle Howells:

I’ve had this problem with 3 different ethernet adapters. If I leave the ethernet USB hub plugged into the ethernet switch it will break the switch sending any network traffic at all unless I unplug it & reboot the switch.

KittensInc:

The interesting thing is that the hub doesn’t seem to be anything explicitly wrong here.

[…]

USB is a bit flaky, and sometimes the connection burps and drops out for a fraction of a second. You don’t really want to lose any data, so you just add a small buffer on the adapter so you can resume when the connection comes back.

But doing that is expensive, and you still run the risk of the buffer filling up. Turns out Ethernet supports a “pause” command which tells the switch to halt sending new data for a bit!

[…]

Besides, it’s completely harmless.

Well, until someone doesn’t really read the Ethernet specs when designing switches. Turns out some switches will honor the pause command, and then forward it to all other device. A blatant violation of the spec, but nobody notices because the pause thing was specified in 1997 and most people forgot about it because it turned out to be pretty useless. And of course the devices they forward it to are honoring the pause commands, so they stop sending data altogether.

15 Comments RSS · Twitter

Wow. This explains all of the issues I ran into with a Hyperdrive dock – I noticed I'd have to unplug the Ethernet cable from the dock when it wasn't connected to a machine or it'd kill access to my entire network. I thought it might be an issue with that specific dock.

I so have this problem and video calls (teams/webex) exacerbate it. When I have a call I unplug the usb ethernet from my laptop and use WiFi.

If I could replace the magsafe on my MBP with an ethernet port (or even a USB A) I would do it without thinking twice.

The USB-C to Gigabit Ethernet Adapter (sold by Belkin with black color; and by Apple with white color) BREAKS file sharing to mount MacBook Pro disk in desktop iMac. Is there a way to fix that?

Mmm, good to know. I haven't enabled 802.3x on my switch ports. I guess if your switches aren't managed you might have trouble preventing propagation though. Arse.

I’ve seen the exact same thing on my hub, except I’ve narrowed it down to when my computer is low on battery. I don’t have any power going into the hub pass through, just Ethernet and hdmi, but maybe it does something similar. either way, I have to unplug the hub from my computer and it seems to clear up the network.

I got bit by a variation of this issue a few weeks ago. I setup my USB-C dock on a smart switch to power it off at night (hopefully to improve my MBP's battery life). I had a USB ethernet adapter plugged into a powered USB3 hub plugged into the dock. Killing power to the dock killed my home network. I plugged the ethernet adapter directly into the hub, so now it shuts off when I kill power to the dock.

[…] is not new, but the topic has surfaced recently In Michael Tsai : USB-C hubs that include an Ethernet port can have an unfortunate tendency to block your entire […]

This only raises more questions!

Is every network switch expected to contain enough buffer for every connected client? When I plug a 100Mbps device into a 1000Mbps network and transfer a 1GB file, is the switch on the last hop supposed to buffer 2 minutes of network traffic for the slow one?

Or is PAUSE intended to be a hint that it's OK to drop frames at the switch? Saving switch-to-adapter network traffic seems like a useless optimization. Even if this were implemented 'correctly', it seems dumb.

Doesn't ethernet (and/or IP, TCP, and probably a layer or two above that) replace lost packets automatically? Why would it need to explicitly pause, when it can simply re-transmit -- which it has to support, anyway?

Why are all these USB devices flooding the network with PAUSE frames when no device is connected? It makes no sense to transmit anything in that case, much less PAUSE frames. Is that even allowed? Do they all use the same network chip, with the same bug? Has this network chip been used before, and if so, is the bug visible in other circumstances (e.g., sleeping desktop computers, whose network circuit would need to be connected and powered for wake-on-LAN)?

Back-pressure is a great concept but this seems to be designed at a point where it can't usefully help. If my computer is getting packets too quickly, I'd really need to signal the original producer of them, not my own switch.

It seems like your headline is inaccurate. It's not the USB-C hubs that are breaking Ethernet, it's bad switches which forward the PAUSE to all devices.

@Andrew The switches are at fault, but the hubs are the trigger.

@Andrew Agreed, but it's hard to see how anybody could fail to anticipate this one. A stupid switch that didn't know or understand 802.3x at all would almost be an outlier not to do that, even though the IEEE specifies that switches that do not participate in 802.3x can simply discard the frames. The good news is that even dumb switches are modernising sufficiently to implement support for newer standards such as 3ad for link aggregation, so hopefully this will eventually be implemented correctly by more vendors. But it does serve to reinforce the need for good protocol design with back-compatibility and implementation simplicity in mind.

@Michael Tsai @algalgal - Thank you so much for this! I just identified an Anker 7-in-1+ that is causing the same problem. RMAd a 12 year old DLINK DGS-1024D (I don't think this was the cause as I was getting individual port slowdowns but not complete drop outs) and replacement switch (refurbished) and replacement to that (new) both ran into this issue within hours. Since my wife wants plug and play for her Macbook Pro I would appreciate any suggestions (currently trying to bypass switch and plugging directly, via patch panel, into Fios G3100 router). Will this work? Are there hubs that are compatible? Are there switches which are compatible?

Thank you for this post. I can't believe that this issue still exists after so many years, and still doesn't seem to be properly resolved. When our network completely crashed for the first time in 2022, it was my partner's USB-C dock (a Lenovo one, I think), which seemed to crash our network every time his notebook went to sleep. After replacing it or after changing network connection, everything seemed to be fine.
Some months later it was me, who bought an Asus SimPro Dock 2 for my own laptop. Soon after that, the crashes were back. They never occurred while I was working, but only some time (often hours) after the shutdown of my notebook. We've even replaced two switches and multiple cables, until we found out that it was the dock, again.
I like Ethernet better than Wi-Fi (it's still faster and usually more stable), so a couple of months later, after one or two firmware upgrades for my Asus dock, I tried again. Everything seemed to be OK for weeks, until recently, when the network crashed again, with my notebook being turned off. Network came back after unplugging the Ethernet cable from my dock.
My thoughts: Nowadays there are more people using USB-C docks, as USB-C is available on every new device. And there are more people who occasionally work from home, who like to easily plug in their notebooks wherever they work. Shouldn't manufacturers try to avoid this kind of hassle? Or are they trying, but not quite succeeding?

Thank you all fo rthis interesting discussion, as i am having the same issue. Have you identified a usb-c ethernet adapater / hub that DOES NOT crash the network? I would love to replace mine. Thank you!

Leave a Comment