The Workstation on top of my server rack has 3 Ethernet ports. One is built in to the mother board, and and two are on a card. I want to use these three ports for different purposes. How can I tell which is which internally? The answer lies in /sys/bus/pci/devices/.
You can get most of what you need if you run nmcli with no arguments.Â
em1: disconnected "Intel 82571EB/82571GB D0/D1" ethernet (e1000e), 00:14:5E:75:F4:DA, hw, mtu 1500 em2: disconnected "Intel 82571EB/82571GB D0/D1" ethernet (e1000e), 00:14:5E:75:F4:DB, hw, mtu 1500 p3p1: disconnected "Realtek RTL8111/8168/8411" 1 connection available ethernet (r8169), C8:1F:66:46:1A:43, hw, mtu 1500 |
Some Output removed.
One important exceptioj is that it does not show the pci data.
ip a shows me this:
$ ip a 2: em1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000 link/ether 00:14:5e:75:f4:da brd ff:ff:ff:ff:ff:ff 3: p3p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether c8:1f:66:46:1a:43 brd ff:ff:ff:ff:ff:ff 4: em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:14:5e:75:f4:db brd ff:ff:ff:ff:ff:ff |
Again, some output removed.
I can see the set of hardware devices from lspci.
$ lspci | grep Ethernet 01:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06) 01:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) (rev 06) 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c) |
The Dual socket card has the two Intel ports. The Realtek is the Motherboard port. But the three address from ip a seem to be all over the place. the p3p1 naming convention implies https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
Or, as the first comment here explains:
” The convention does actually make a little sense as it now names network ports on the motherboard as interface “emX” where X is a digit starting at 0. It then names ethernet ports that are on plugin cards as pYpZ where Y is the PCI slot number and Z is the port number on that card. It’s meant to make the naming of devices more consistent and to alleviate the occurrences of people complaining that their eth0 is now eth1 etc.
I do not understand why I have an em2 device, though. I wonder if I delete it, and reboot, if it will show up as an p3p* name instead? Lets find out.
2: p3p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether c8:1f:66:46:1a:43 brd ff:ff:ff:ff:ff:ff 3: em1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000 link/ether 00:14:5e:75:f4:da brd ff:ff:ff:ff:ff:ff 4: em2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000 link/ether 00:14:5e:75:f4:db brd ff:ff:ff:ff:ff:ff |
Nope. And…only the motherboard is plugged in right now, and that is the link showing as Up, so p3p1 is the MoBo port, and the em* ports are the Intel. But…what if we did not have physical access, as it likely the case when working with a machine in a datacenter?
In general, linux systems have an enumeration of devices under their bus type in the /sys directory. If run lspci, it is enumerating the sub-directories of
$ ls /sys/bus/pci/devices/ 0000:00:00.0 0000:00:02.0 0000:00:14.0 0000:00:1a.0 0000:00:1c.0 0000:00:1c.4 0000:00:1f.0 0000:00:1f.3 0000:01:00.1 0000:04:00.0 0000:00:01.0 0000:00:03.0 0000:00:16.0 0000:00:1b.0 0000:00:1c.3 0000:00:1d.0 0000:00:1f.2 0000:01:00.0 0000:03:00.0 |
In much the same way the lsusb lists:
# ls /sys/bus/usb/devices/ 1-0:1.0 1-1 1-1:1.0 2-0:1.0 2-1 2-1:1.0 3-0:1.0 3-10 3-10:1.0 3-10:1.1 3-2 3-2:1.0 3-2:1.1 3-6 3-6:1.0 3-7 3-7:1.0 4-0:1.0 usb1 usb2 usb3 usb4 |
However, network devices are managed via a socket:
[root@dialga ~]$ strace ip a 2>&1 | grep open open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 open("/etc/iproute2/group", O_RDONLY) = 4 [root@dialga ~]$ strace ip a 2>&1 | grep socket socket(AF_NETLINK, SOCK_RAW|SOCK_CLOEXEC, NETLINK_ROUTE) = 3 |
And this is what nmcli uses.
What if we wanted to script this, and match on the ids? ethtool pulls out the pci id. See the bus-info field:
$ ethtool -i p3p1 driver: r8169 version: 2.3LK-NAPI firmware-version: rtl8168g-2_0.0.1 02/06/13 expansion-rom-version: bus-info: 0000:03:00.0 supports-statistics: yes supports-test: no supports-eeprom-access: no supports-register-dump: yes supports-priv-flags: no |
In this case it is 0000:03:00.0 which is the name of the subdir under /sys/bus/pci/devices/
$ ls /sys/bus/pci/devices/0000:03:00.0 broken_parity_status consistent_dma_mask_bits dma_mask_bits enable local_cpulist msi_bus numa_node rescan resource0 resource4_wc subsystem_vendor vpd class d3cold_allowed driver firmware_node local_cpus msi_irqs power reset resource2 subsystem uevent config device driver_override irq modalias net remove resource resource4 subsystem_device vendor |
And notice that there is a subdirectory:
$ ls /sys/bus/pci/devices/0000:03:00.0/net p3p1 |
However…find does not work in this subtree:
$ find /sys/bus/pci/devices/ -name net |
The p3p1 subdirectory has all of the Kernel exposed data about the network device that you are likely to need.
You can also use the other way around:
readlink -f /sys/class/net/p3p1/device
find in your last example does not work because of the symlinks
I use “udevadm test” to check what caused a netdev to be named what it is:
udevadm test /sys/class/net/p3p1
My suspicion is that you have “biosdevname” enabled, meaning that systemd names are overridden. (I believe biosdevname assumes that the smbios is authoritative). You can double-check by booting with biosdevname=0 on the kernel cmdline.
If you see “eno1” and “ens3XX” it’s likely systemd/udev naming.