Predictably non-persistent names

This is going to be fun. The Gentoo “udev team”, in the person of Samuli – who seems to suffer from 0-day bump syndrome – decided to now enable by default the new predictable names feature that is supposed to make things so much nicer in Linux land where, especially for people coming from FreeBSD, things have been pretty much messed up. This replaces the old “persistent” names, that were often enough too fragile to work, as they did in-place renaming of interfaces, and would cause way too often conflicts at boot time, since swapping two devices’ names is not an atomic operation for obvious reasons.

So what’s this predictable name all around? Well, it’s mostly a merge of the previous persistent naming system, and the BIOS label naming project which was developed by RedHat for a few years already so that the names of interfaces for server hardware in the operating system match the documentation of said server, so that you can be sure that if you’re connecting the port marked with “1” on the chassis, out of four on the motherboard, it will bring up eth2.

But why were those two technologies needed? Let’s start first with explaining how (more or less) the kernel naming scheme works: unlike the BSD systems, where the interfaces are named after the kernel driver (en0, dc0, etc.), the Linux kernel uses generic names, mostly eth, wlan and wwan, and maybe a couple more, for tunnels and so on. This causes the first problem: if you have multiple devices of the same class (ethernet, wlan, wwan) coming from different drivers, the order of the interface may very well vary between reboots, either because of changes in the kernel, if the drivers are built-in, or simply because of locking and execution of modules load (which is much more common for binary distributions).

The reason why changes in the kernel can change the order is that the order in which drivers are initialized has changed before and might change again in the future. A driver could also decide to change the order with which its devices are initialized (PCI tree scanning order, PCI ID order, MAC address order, …) and so on, causing it to change the order of interfaces even for the same driver. More about this later.

But here’s my first doubt arises: how common is for people to have more than one interface of the same class from vendors different enough to use different drivers? Well it depends on the class of device; on a laptop you’d have to search hard for a model with more than one Ethernet or wireless interface, unless you add an ExpressCard or PCMCIA expansion card (and even those are not that common). On a desktop, I’ve seen a few very recent motherboards with more than one network port, and I have yet to see one with different chips for the two. Servers, that’s a different story.

Indeed, it’s not that uncommon to have multiple on-board and expansion card ports on a server. For instance you could use the two onboard ports as public and private interfaces for the host… and then add a 4-port card to split between virtual machines. In this situation, having a persistent naming of the interfaces is indeed something you would be glad of. How can you tell which one of eth{0..5} is your onboard port #2, otherwise? This would be problem number two.

Another situation in which having a persistent naming of interfaces is almost a requirement is if you’re setting up a router: you definitely don’t want to switch the LAN and WAN interface names around, especially where the firewall is involved.

This background is why the persistent-net rules were devised quite a few years ago for udev. Unfortunately almost everybody got at least one nasty experience with them. Sometimes the in-place rename would fail, and you’d end up with the temporary names at the end of boot. In a few cases the name was not persistent at all: if the kernel driver for the device would change, or change name at least, the rules wouldn’t match and your eth0 would become eth1 (this was the case when Intel split the e1000 and e1000e drivers, but it’s definitely more common with wireless drivers, especially if they move from staging to main).

So the old persistent net rules were flawed. What about the new predictable rules? Well, not only they combined the BIOS naming scheme (which is actually awesome when it works — SuperMicro servers such as Excelsior do not expose the label; my Dell laptop only exposes a label for the Ethernet port but doesn’t for either the wireless adapter or the 3G one), but it has two “fallbacks” that are supposed to be used when the labels fail, one based on the MAC address of the interface, and the other based on the “path” — which for most PCI, PCI-E, onboard, ExpressCard ports is basically the PCI address; for USB… we’ll see in a moment.

So let’s see, from my laptop:

# lspci | grep 'Network controller'
03:00.0 Network controller: Intel Corporation Centrino Advanced-N 6200 (rev 35)
# ifconfig | grep wlp3
wlp3s0: flags=4163  mtu 1500

Why “wlp3s0”? It’s the Wireless adapter (wl) PCI (p) card at bus 3, slot 0 (s0): 03:00.0. Matches lspci properly. But let’s see the WWAN interface on the same laptop:

# ifconfig -a | grep ww
wwp0s29u1u6i6: flags=4098  mtu 1500

Much longer name! What’s going on then? Let’s see, it’s reporting it’s card at bus 0, slot 29 (0x1d) — lspci will use hexadecimal numbers for the addresses:

# lspci | grep '00:1d'
00:1d.0 USB controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 05)

Okay so it’s an USB device, even though the physical form factor is a mini-PCIE card. It’s common. Does it match lsusb?

# lsusb | grep Broadband
Bus 002 Device 004: ID 413c:8184 Dell Computer Corp. F3607gw v2 Mobile Broadband Module

Not the Bus/Device specification there, which is good: the device number will increase every time you pop something in/out of the port, so it’s not persistent across reboots at all. What it uses is the path to the device standing by USB ports, which is a tad more complex, but basically means it matches /sys/bus/usb/devices/2-1.6:1.6/ (I don’t pretend to know how the thing works exactly, but it describe to which physical port the device is connected).

In my laptop’s case, the situation is actually quite nice: I cannot move either the WLAN or WWAN device on a different slot so the name assigned by the slot is persistent as well as predictable. But what if you’re on a desktop with an add-on WLAN card? What happens if you decide to change your video card, with a more powerful one that occupies the space of two slots, one of which happen to be the place where you WLAN card is? You move it, reboot and .. you just changed the interface name! If you’ve been using Network Manager, you’ll just have to reconfigure the network I suppose.

Let’s take a different example. My laptop, with its integrated WWAN card, is a rare example; most people I know use USB “keys”, as the providers give them away for free, at least in Italy. I happen to have one as well, so let me try to plug it in one of the ports of my laptop:

# lsusb | grep modem
Bus 002 Device 014: ID 12d1:1436 Huawei Technologies Co., Ltd. E173 3G Modem (modem-mode)
# ifconfig -a | grep ww
wwp0s29u1u2i1: flags=4098  mtu 1500
wwp0s29u1u6i6: flags=4098  mtu 1500

Okay great this is a different USB device, connected to the same USB controller as the onboard one, but at different ports, neat. Now, what if I had all my usual ports busy, and I decided to connect it to the USB3 add-on ExpressCard I got on the laptop?

# lsusb | grep modem
Bus 003 Device 004: ID 12d1:1436 Huawei Technologies Co., Ltd. E173 3G Modem (modem-mode)
# ifconfig -a | grep ww
wwp0s29u1u6i6: flags=4098  mtu 1500
wws1u1i1: flags=4098  mtu 1500

What’s this? Well, the USB3 controller provides slot information, so udev magically uses that to rename the interface, so it avoids using the otherwise longer wwp6s0u1i1 name (the USB3 controller is on the PCI bus 6).

Let’s go back to the on-board ports:

# lsusb | grep modem
Bus 002 Device 016: ID 12d1:1436 Huawei Technologies Co., Ltd. E173 3G Modem (modem-mode)
# ifconfig -a | grep ww
wwp0s29u1u3i1: flags=4098  mtu 1500
wwp0s29u1u6i6: flags=4098  mtu 1500

Seems the same, but it’s not. Now it’s u3 not u2. Why? I used a different port on the laptop. And the interface name changed. Yes, any port change will produce a different interface name, predictably. But what happens if the kernel decides to change the way the ports are enumerated? What happens if the USB 2 driver is buggy and is supposed to provide slot information, and they fix it? You got it, even in these cases, the interface names are changed.

I’m not saying that the kernel naming scheme is perfect. But if you’re expected to always only have an Ethernet port, a WLAN card and a WWAN USB stick, with it you’ll be sure to have eth0, wlan0 and wwan0, as long as the drivers are not completely broken as they are now (like if the WLAN is appearing as eth1), and as long as you don’t muck with the interface names in userspace.

Next up, I’ll talk about the MAC addresses based naming and my personal preference when setting up servers and routers. Have fun in the mean time figuring out what your interface names will be.

9 thoughts on “Predictably non-persistent names

  1. The FreeBSD model you like so much is pretty broken too. Such a model is naive with modern hw where devices are probed in parallel and probing is unbounded by time, and you might easily have different hw controlled by the same driver… Binding naming to drivers is only useful where people never place two or more identical devices in the same machine…Red Hat didn’t write biosdevname (it’s a Dell thing), and the persistent stuff in systemd/udev shares no code with it.BTW, just plug in your smartphone and you immediately do have another ethernet iface on your laptop. If you have a broadcom wlan device you have multiple interfaces on it anyway.The ids used in the names for USB devices are the “stable” bits of a USB path, i.e. on the USB descriptor level they are exposed as properties of the physical layout of the chip. i.e. as long as you plug your USB device into the same USB socket every time it will have the exact same name.We specifically bind names to the “stable” bits of device paths, since we want to allow people to replace broken hw. As long as you plug things in at the same physical location the naming won’t change. If you plug in your hw into another location the name will change, however. If you prefer binding the naming to actual specific devices you bought, you can choose to name things after the MAC address, but that’s just an option. In general we chose to default to fixed-by-path rather than fixed-by-identity since we had to pick some default and it appeared to be smarter to allow replacing bad hw than anything else. (Especially since MAC addresses aren’t as stable as they used to be either…)What this new approach does that previous approaches didn’t is that we never make up ID information, i.e. all we include in the names are IDs passed to us from firmware or from hardware chips. These aren’t enumeration-order-dependent IDs assigned by drivers, but IDs assigned by the chips and firmware itself. That means updating your kernel, your drivers, your udev, your OS should not reorder things, and that’s what we think is important. However, firmware updates might in very few circumstances result in name changes. So this really isn’t about making things perfectly stable and predictable under all conditions thinkable — but do the best we can and at least never make up IDs of our own ever. Note that the firmware information is usually provided primarily to make names stable, so BIOS vendors tend to keep them pretty stable — but then again, BIOS vendors suck, so of course, this might not work. BTW, even cheaper HP or Lenovo laptops have this kind of firmware info these days.

    Like

  2. I always wondered why it isn’t possible to identify a eth interface via its MAC address. Just like block devices can be identified by UUID.

    Like

  3. I could have sworn to have seen RedHat people working around biosdevname — and basically most of its features seem to be folded in this one, so for what I’m concerned, it’s superseding it.For your information, HP servers at generation 6, current generation SuperMicro servers, two-generations-ago Dell laptops and if I’m not mistaken my ZenBook as well all do not provide (complete) firmware-level labels, so the general case is far from being completely covered by the BIOS names!So you say that “updating your kernel, your drivers, your udev, your OS should not reorder things”. The udev part is to be seen, since so many stability promises from udev and company have been broken before. Which leaves you with at least three instability causes: port/slot, udev, and firmware. The first in particular for USB stuff is terrible.What you get at that point is that, on a laptop or desktop with usb interfaces, your network names _will_ be almost completely unpredictable unless you switch to MAC based. But of course desktop users don’t care, as they use NetworkManager…… but if they don’t care if it’s wwp0s29u1u2i1 or wwp0s29u1u3i1 why should they care if it’s wwan0 or wwan124?

    Like

  4. I’ve seen a lot of servers (mainly Dell and HP) with RHEL4-5 installed and there was no single case when devices naming could cause problems. Their ifcfg-eth# files have the MACADDR option so it just worked well as it was expected.As for desktops – user should never see these weird names. Thes should be “Internal Wifi card #0”, “External usb 3G modem #1” and so on. wwp0s29u1u2i1 is why linux will never win desktop.

    Like

  5. What I’m wondering, is how this “device path”-based naming system will work on VMs running in paravirtual mode on top of Xen…

    Like

  6. Hah, we have servers sporting two different brands of LAN chip *soldered to the motherboard*. I think one has an Intel 10/100 chip and a Broadcom Gb chip for its two built-in NICs.

    Like

  7. The fact that you can identify interfaces by MAC address just goes to show how stupid and pointless this whole ‘persistent, predictable’ naming scheme is–it wasn’t broken, why the hell do we need this ‘fix’?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s