How Debian managed the systemd transition [LWN.net]

How Debian managed the systemd transition

Posted Sep 18, 2015 2:19 UTC (Fri) by flussence (guest, #85566)
Parent article: How Debian managed the systemd transition

> The first change is to udev, which will begin assigning predictable, stable names for network interfaces (in place of names using the ambiguous "eth0" form).

I hope, for all Debian users' sake, that they've gone an extra mile flushing out all the bugs in that.

I've seen quite a few instances (in Gentoo, which adopted it years ago) where someone's network setup was broken due to it: bus enumeration races causing a USB wifi dongle to change names every boot, the rules flip-flopping between bus and MAC naming patterns between udev versions, the "en"/"wl" prefix sometimes going missing entirely, and even multiple of these problems striking at once during the jump from install media to finished system.

In ten years of using it and participating in their help forums, I've yet to come across a *single* complaint that eth0 unexpectedly became eth1. Maybe Gentoo users simply lack sufficiently insane hardware?

to post comments

How Debian managed the systemd transition

Posted Sep 18, 2015 3:02 UTC (Fri) by dlang (guest, #313) [Link] (17 responses)

predictable and stable if you happen to have the mac address of every interface memorized.

If my laptop only has one ethernet interface, "eth0" is predictable and stable.

enx00249b0e398f is neither (and yes, this is exactly what the new udev gave me for my laptop)

Now, I have managed a lot of systems with lots of interfaces over the years (22 gig-e interfaces on one system was one of my 'standard' configs).

I have had a couple cases where interface names have changed during kernel upgrades.

In one case it was because a new driver enabled an interface that previously had been unsupported

In the other case, it was because a different driver was used to support one card and it got sorted ahead of the other driver supporting other cards.

I can understand that people who do everything with modules loaded at runtime can run into grief by loading the modules in a different order, but this falls under the category of 'it hurts when I do that, so stop doing that'

Debian has for years supported pinning the interface names to the MAC addresses based on the detection order the first time the OS booted. This has actually caused me more grief than anything else, because when I replaced the card that held eth0-eth3 with an identical card, now I had eth22-eth25 and no eth0-eth3 (until I removed the udev rule that did this)

If you can't tell, I think the new udev 'solution' is far worse than any problem it may solve.

the good news is that it's fairly easy to disable

# ln -s /dev/null /etc/udev/rules.d/80-net-setup-link.rules

unfortunately this doesn't give me eth0 back, it's now usb0, but at least I don't have to remember the 12 digit hex id!

How Debian managed the systemd transition

Posted Sep 18, 2015 13:31 UTC (Fri) by epa (subscriber, #39769) [Link] (12 responses)

Maybe a halfway house would be device name 'eth_only' when there is exactly one card in the machine (a fairly common case). It would never change as long as there is only one. If you installed a second Ethernet adaptor then you would be forced to switch over to the funky udev names or else use eth0 and eth1 with some risk that they might be reordered for some random reason.

How Debian managed the systemd transition

Posted Sep 18, 2015 17:26 UTC (Fri) by kreijack (guest, #43513) [Link] (11 responses)

I think that a more reasonable solution is to still provide the ethN interface name as alias of the new name. The ethN interfaces may depend by the discovery order, but for the common case (a pc with only one ethernet) it avoids the problem.

BTW I never seen a swap of the ethernet cards; may be that I use only PVI ones ?

How Debian managed the systemd transition

Posted Sep 18, 2015 18:13 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (5 responses)

> I think that a more reasonable solution is to still provide the ethN interface name as alias of the new name.
Network code doesn't support it and network devices have always been a 'special case' in Linux. Notice how there are no '/dev/ethX' files.

How Debian managed the systemd transition

Posted Sep 19, 2015 14:34 UTC (Sat) by cortana (subscriber, #24596) [Link] (3 responses)

I wonder why no one has ever looked into increasing the maximum network interface name length, and adding a new kind of 'n' device node that network interfaces can use, so that they can show up in /dev just like other devices.

Then we could have /dev/net/by-{index,slot,path,mac} symlinks pointing to the original /dev/eth0 and friends. Both the original names and the predictable names would be there for those who use them, and all names would be usable simultaneously.

That way, so you no longer have to worry about a network interface having its name changed from eth0 to enp0s3 when you reboot with a newer version of udev, as happened to me last week.

You would also be able to pick and choose which name you want to use depending on your use case, just as how, with disks, it's sometimes useful to use the symlink in /dev/disk/by-path for a disk that is commonly hot-swapped (such as a backup disk you change every week) but at the same time it is useful to use /dev/disk/by-uuid for a permanently-mounted filesystem, such as / or /home.

How Debian managed the systemd transition

Posted Sep 19, 2015 20:40 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

> I wonder why no one has ever looked into increasing the maximum network interface name length, and adding a new kind of 'n' device node that network interfaces can use, so that they can show up in /dev just like other devices.
As usual - legacy. The interface name is limited by IFNAMSIZ value which is used in multiple contexts, so you can not use long strings like '/dev/ethN' for interface names. And also there are no provisions for multiple interface names in getifaddrs() calls or similar functions.

Adding new functions that work with regular device nodes is certainly possible, but probably nobody cares too much about it.

How Debian managed the systemd transition

Posted Sep 20, 2015 0:02 UTC (Sun) by dlang (guest, #313) [Link] (1 responses)

if they can do eth<mac> then /dev/ethN isn't unreasonably long

How Debian managed the systemd transition

Posted Sep 20, 2015 7:37 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

IFNAMSIZ is equal to 16, so you only have 15 letters for the full path.

And the second problem is that you don't actually _need_ your devices to be mounted at '/dev'. A device node can be anywhere on the system.

It doesn't even require anything exotic - a host might simply with to get info about a TAP/TUN device in a container.

The correct way to fix it would be by creating a new API that uses file descriptors instead of interface names.

How Debian managed the systemd transition

Posted Sep 20, 2015 10:29 UTC (Sun) by kreijack (guest, #43513) [Link]

> > I think that a more reasonable solution is to still provide the ethN interface name as alias of the new name.
> Network code doesn't support it and network devices have always been a 'special case' in Linux.
> Notice how there are no '/dev/ethX' files.

May be that *now* doesn't exist a technical solution for doing that. But I don't see any reason which prevent us to implement it.
Let me to rephrase my sentence: my suggestion could be cause of some problem, or today it is impossible because nobody had coded that solution.
The former may/is a stop, the latter could be solved by some hours of coding (and several days of testing :-) )

How Debian managed the systemd transition

Posted Sep 19, 2015 1:34 UTC (Sat) by dlang (guest, #313) [Link] (4 responses)

> BTW I never seen a swap of the ethernet cards;

a swap can happen if you have cards that use different drivers and the driver load order changes (either from driver renaming if the drivers are statically linked, or the device discovery order if they are modules)

a swap can also happen if the bus the cards are on does not have an order and orders based on the order devices respond. I'm told that USB can detect the physical location of a device, but asaik, all the drivers ignore this and rely on which device responds first to device probing.

But if the device has a MAC address as part of the device, the existing tools can keep the ordering consistant.

If there is no such identifier built into the card, I don't believe that the new process is really any more reliable.

How Debian managed the systemd transition

Posted Sep 20, 2015 10:22 UTC (Sun) by kreijack (guest, #43513) [Link] (3 responses)

> a swap can happen if you have cards..
What you are saying is correct.... But I repeat in my (very limited) experience I never seen that.
What I see is that the ethernet name change from eth0 to eth1 when I moved from a disk image from one host to another, and that cause me more headache then the fact that I have more ethernet device of different hardware...

What I mean is that linking the ethernet name to the hardware is useful in some contexts, but in others not; and I suspect that these "others" are more common than the former...

How Debian managed the systemd transition

Posted Sep 20, 2015 10:52 UTC (Sun) by dlang (guest, #313) [Link]

I agree, I always disable that on my systems. But I can see that if someone was dealing with multiple USB interfaces, or wants to do async module loading (with all the ordering race conditions it introduces) in search of faster boot times, I can see why it could be useful.

How Debian managed the systemd transition

Posted Sep 25, 2015 20:16 UTC (Fri) by Wol (subscriber, #4433) [Link] (1 responses)

> What I mean is that linking the ethernet name to the hardware is useful in some contexts

You mean, like in a firewall, for instance ...

I saw a comment somewhere where eth0 and eth1 got swapped. In other words, until someone noticed, the firewall's soft, unprotected, meant for the internal network, interface was the interface to the hostile outside world ...

Cue major panic, lan disconnected from the internet, rebuild the firewall from scratch, ...

At the end of the day, unpredictable behaviour is a security risk. What I think happened was that the system had always been booted from cold, and had always been predictable. Then for some reason, one day it did a warm-start, and came back with the interfaces swapped over ... oops ...

Cheers,
Wol

How Debian managed the systemd transition

Posted Sep 25, 2015 20:36 UTC (Fri) by dlang (guest, #313) [Link]

actually, if the firewall interfaces get swapped, it's not going to talk to anything, because it is going to send the reply packets out the wrong interface (unless you also have dynamic routing)

Yes, it is possible to configure a system so that it's IPs and routes will swap with the interface changes, but the firewall rules won't, but that's getting into rather contrived territory.

How Debian managed the systemd transition

Posted Sep 18, 2015 16:42 UTC (Fri) by HenrikH (subscriber, #31152) [Link] (3 responses)

Actually the current pinning seams to be a much better solution that creating a dev with the mac as name. Just as long as one rembers to edit /etc/udev/rules.d/70-persistent-net.rules if one have changed the NIC. Now with enx00249b0e398f one would possible have to edit tons and tons of config files if unlucky.

How Debian managed the systemd transition

Posted Sep 19, 2015 18:08 UTC (Sat) by cortana (subscriber, #24596) [Link] (2 responses)

The problem with the old 'pinning' approach was that it was racey. I have had to recover servers that stopped working because the interface usually known as eth0 is now called eth1_rename, and there is another interface now called eth0, or some such.

How Debian managed the systemd transition

Posted Sep 20, 2015 0:04 UTC (Sun) by dlang (guest, #313) [Link] (1 responses)

Interesting, i never ran into that.

It may be that I got bit by the rename due to card changes and disabled it entirely before the odds caught up with me.

How Debian managed the systemd transition

Posted Sep 20, 2015 13:14 UTC (Sun) by cortana (subscriber, #24596) [Link]

It is a probabilistic problem; you might never see it in your whole career; or you might see it twice in a week. It is annoying when those who have never seen it happen to them (and I am not including you in this set) assume that it is not a problem, and rubbish the efforts of those who are trying to fix it properly by removing the underlying race condition.

How Debian managed the systemd transition

Posted Sep 18, 2015 15:59 UTC (Fri) by lsl (subscriber, #86508) [Link] (1 responses)

Something in Debian sid recently felt the need to rename a virtio device from eth0 to ens3 with the result that the VM was offline until someone logged in on the console to do a s/eth0/ens3/. No idea if that is a bug or just the normal risk of running unstable.

How Debian managed the systemd transition

Posted Sep 18, 2015 19:12 UTC (Fri) by lsl (subscriber, #86508) [Link]

It apeears that this was indeed considered a bug and rolled-back with a new update. The device regained its eth0 name, so the VM is once again unreachable. Thankfully, crap like this is easy to deal with on virtualized systems.

How Debian managed the systemd transition

Posted Sep 20, 2015 12:09 UTC (Sun) by patrakov (subscriber, #97174) [Link]

Well, I could not say "eth0 became eth1", but I had a case when eth1 became eth2. On Debian.

How Debian managed the systemd transition

Posted Oct 5, 2015 14:22 UTC (Mon) by Creideiki (subscriber, #38747) [Link]

Another problem is that while the interface names might (sometimes) be predictable across time on a single machine, they are emphatically not predictable across several machines at the same time. Especially with hardware manufacturers changing how they wire up their PCI buses without changing anything other than the revision number printed on the motherboard.

I've had a batch of computers, all the same nominal model from the same (big) manufacturer, all installed from the same netboot image, half calling their only Ethernet interface one thing and half calling it another. System administration scripting was so much easier when all machines had an eth0 connecting to the main network.