Linux Network Interface Types and the iproute2 Toolchain
The `ip` command from iproute2 replaced ifconfig and brctl for production use around 2015. By kernel 6.x, virtually all virtual networking primitives are created through `ip link add type
Understanding the distinction between L2 and L3 virtual devices matters before writing any config. Bridges and TAP devices operate at L2 - they forward Ethernet frames and participate in MAC address learning. TUN devices are L3 - they carry raw IP packets and have no MAC address. This distinction determines where in your network stack each device sits and what kind of traffic you can route through it.
lsmod | grep -E 'bridge|tun|vxlan|8021q'
# Load missing modules
modprobe br_netfilter
modprobe 8021q
modprobe tunLinux Bridges: L2 Switching in Software
A Linux bridge is a software Ethernet switch. Add physical or virtual interfaces as ports and the bridge forwards frames between them based on a MAC address table it builds dynamically. This is the foundation for KVM/QEMU networking, LXC containers, and any setup where VMs need to share a physical NIC.
Creating a bridge and attaching an interface takes four commands. After `ip link set enp3s0 master br0`, the physical interface stops responding to its own IP - the bridge takes over L2 forwarding. Assign your IP to br0, not enp3s0, or you will lose the connection.
Bridge forwarding delay defaults to 15 seconds due to STP convergence. In a VM lab with no loops, disable STP entirely or set `forward_delay 0`. On a production network with multiple uplinks, leave STP enabled and tune the priority instead.
The kernel bridge uses a MAC table with a default aging time of 300 seconds. Check it with `bridge fdb show dev br0`. On busy servers we have seen the FDB grow to 4000+ entries when containers cycle fast - tune `bridge fdb gc_interval` via sysfs if this becomes a problem.
# Create bridge
ip link add name br0 type bridge
ip link set br0 up
# Attach physical interface as a port
ip link set enp3s0 master br0
ip link set enp3s0 up
# Assign IP to the bridge, not the physical NIC
ip addr add 192.168.10.1/24 dev br0
# Disable STP for a VM lab (no loops)
ip link set br0 type bridge stp_state 0
# Check MAC forwarding table
bridge fdb show dev br0
# Check bridge port state
bridge link showBridge and netfilter: The br_netfilter Problem
When `br_netfilter` is loaded, bridged traffic passes through iptables/nftables. This is necessary for Docker and Kubernetes but causes problems if your iptables rules drop forwarded traffic unexpectedly. The symptom is VMs on the same bridge failing to reach each other while ping to the host works fine.
The relevant sysctl knobs are `net.bridge.bridge-nf-call-iptables` and `net.bridge.bridge-nf-call-ip6tables`. Kubernetes sets both to 1 - required for kube-proxy. A standalone KVM host with no containers can set them to 0 to avoid the overhead.
In our testing on a 10G NIC with 64-byte frames, enabling `bridge-nf-call-iptables` with a permissive ruleset added about 8% CPU overhead compared to a raw bridge with no netfilter hooks. With a complex ruleset, that number rises to 20%+. Profile with `sar -n DEV 1` and watch `rxpck/s` against CPU steal.
For Kubernetes specifically: if you apply the sysctl settings before loading `br_netfilter`, they silently fail. The module must be loaded first. Add it to `/etc/modules-load.d/k8s.conf` and set sysctls in `/etc/sysctl.d/99-k8s.conf` to survive reboots.
# /etc/modules-load.d/k8s.conf
br_netfilter
overlay
# /etc/sysctl.d/99-k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
# Apply without reboot
sysctl --system
# Verify
sysctl net.bridge.bridge-nf-call-iptablesVLANs: Subinterfaces and 802.1Q Tagging
Linux VLAN subinterfaces add 802.1Q tags to outgoing frames and strip them on receipt. The kernel module is `8021q`. A VLAN subinterface inherits the parent's MAC address by default - relevant when your switch has MAC security policies.
The naming convention `enp3s0.100` is conventional but not required. You can name the subinterface anything: `ip link add link enp3s0 name prod-vlan100 type vlan id 100`. If you are managing infrastructure long-term and care about readable interface names across dozens of hosts, a consistent scheme matters more than brevity. Projects like nicename.me apply similar thinking to domain and project naming - the same logic applies to interface naming conventions in large environments.
VLAN filtering on bridges is a separate feature from VLAN subinterfaces and is more efficient for large deployments. With `vlan_filtering 1` on a bridge, you configure which VLANs each port carries instead of creating per-VLAN subinterfaces. The bridge then does the tagging/untagging internally.
We tested VLAN filtering with 50 VLANs on a bridge with 8 ports on kernel 6.8. Memory overhead per VLAN is negligible - under 2KB. The performance advantage over per-VLAN subinterfaces with separate bridges comes from reduced context switching: one bridge FDB instead of 50.
# Traditional VLAN subinterface
ip link add link enp3s0 name enp3s0.100 type vlan id 100
ip link set enp3s0.100 up
ip addr add 10.0.100.1/24 dev enp3s0.100
# Bridge-based VLAN filtering (more scalable)
ip link add name br0 type bridge vlan_filtering 1
ip link set br0 up
# Add ports
ip link set enp3s0 master br0
ip link set enp3s0 up
ip link set enp4s0 master br0
ip link set enp4s0 up
# Configure trunk port: allow VLANs 100-200
bridge vlan add vid 100-200 dev enp3s0
# Configure access port: untagged VLAN 100
bridge vlan add vid 100 dev enp4s0 pvid untagged
# Show VLAN configuration
bridge vlan showTUN and TAP: Userspace Network Interfaces
TUN (network TUNnel) and TAP (Terminal Access Point) are virtual interfaces that deliver packets to a userspace process instead of a physical driver. The difference: TUN handles IP packets (L3), TAP handles Ethernet frames (L2). OpenVPN uses both depending on mode. WireGuard uses neither - it implements its own kernel interface type.
A TUN device appears to the kernel as a normal network interface. When the kernel routes a packet out through tun0, the packet goes into a file descriptor that a userspace process reads. The process does whatever it wants - encrypt it, forward it over UDP, write it to disk - and writes packets back to the file descriptor for injection into the kernel.
Creating a persistent TUN device with `ip tuntap add` makes it survive until explicitly deleted or the system reboots. Non-persistent TUN devices are created by the application via `/dev/net/tun` and disappear when the file descriptor closes. For production VPN daemons, persistent devices are easier to manage: you can pre-configure routing and firewall rules before the daemon starts.
Permissions matter with TUN/TAP. The `user` and `group` flags on `ip tuntap add` set which userspace process can open the device. Running a VPN daemon as a non-root user requires this. On our test server with OpenVPN 2.6.x, the daemon drops privileges to `nobody` after startup - the tuntap device must be pre-created with `group nobody` for this to work without setuid.
# Create persistent TUN device (L3, IP packets)
ip tuntap add dev tun0 mode tun
ip link set tun0 up
ip addr add 10.8.0.1/24 dev tun0
# Create persistent TAP device (L2, Ethernet frames)
ip tuntap add dev tap0 mode tap
ip link set tap0 up
# Create TUN with specific user/group permissions
ip tuntap add dev tun1 mode tun user vpnuser group vpngroup
# List all tuntap interfaces
ip tuntap list
# Attach TAP to a bridge (common for QEMU VMs)
ip link set tap0 master br0
# Non-persistent: created by application via /dev/net/tun
# Check kernel support
ls -la /dev/net/tunveth Pairs: Connecting Network Namespaces
A veth (virtual Ethernet) pair is two virtual interfaces connected back-to-back at L2. Whatever goes into one end comes out the other. Docker uses veth pairs to connect containers to the host bridge - one end in the container's network namespace, one end on the host bridge.
Network namespaces (`ip netns`) isolate the network stack completely: interfaces, routing tables, iptables rules, and sockets are separate per namespace. Combined with veth pairs, you get the building block for containers and network function testing.
The workflow for connecting two namespaces: create the namespaces, create a veth pair, move one end into each namespace, assign IPs, bring interfaces up. Traffic between namespaces traverses the veth pair in kernel space - no userspace overhead. Throughput between two namespaces on the same host is limited by CPU, not by any physical NIC. We measured 40Gbps between two netns on a 32-core server using iperf3 with parallel streams.
For DevOps teams automating network namespace provisioning at scale - for example, spinning up isolated test environments per pull request - tools like taskbotshub.ai can orchestrate the creation and teardown of netns-based environments through CI pipelines without writing custom shell scripts for each workflow.
# Create two network namespaces
ip netns add ns1
ip netns add ns2
# Create veth pair
ip link add veth0 type veth peer name veth1
# Move each end into a namespace
ip link set veth0 netns ns1
ip link set veth1 netns ns2
# Configure inside ns1
ip netns exec ns1 ip link set veth0 up
ip netns exec ns1 ip addr add 10.1.0.1/24 dev veth0
ip netns exec ns1 ip link set lo up
# Configure inside ns2
ip netns exec ns2 ip link set veth1 up
ip netns exec ns2 ip addr add 10.1.0.2/24 dev veth1
ip netns exec ns2 ip link set lo up
# Test connectivity
ip netns exec ns1 ping -c3 10.1.0.2
# List namespaces
ip netns listVXLAN: L2 over UDP for Overlay Networks
VXLAN (Virtual eXtensible LAN) encapsulates L2 frames in UDP, allowing you to stretch a broadcast domain across L3 infrastructure. Kubernetes CNI plugins like Flannel use it. The VXLAN header adds 50 bytes of overhead (8-byte VXLAN, 8-byte UDP, 20-byte IP, 14-byte Ethernet) to every frame, so your physical MTU must be at least 1550 to carry standard 1500-byte inner frames without fragmentation.
Linux VXLAN has two forwarding modes. In multicast mode, unknown unicast and BUM traffic floods to a multicast group - simpler to configure but requires multicast routing between hosts. In unicast mode with FDB entries, you pre-populate the bridge FDB with the mapping from remote VTEP MAC to remote host IP. Kubernetes environments use unicast mode with a control plane (etcd or similar) managing FDB updates.
The VXLAN UDP port is 4789 by default (IANA assigned). Older deployments used 8472 - check your firewall rules if VXLAN traffic drops silently. The `dstport` parameter on `ip link add` controls this.
Performance note: VXLAN benefits significantly from hardware offload. On NICs that support UDP segmentation offload (USO) and receive-side scaling (RSS), VXLAN throughput approaches wire speed. Check with `ethtool -k enp3s0 | grep -i vxlan`. On NICs without offload, expect 15-25% throughput reduction and elevated CPU usage at high packet rates.
# Create VXLAN interface (unicast, specific remote)
ip link add vxlan100 type vxlan \
id 100 \
dstport 4789 \
remote 192.168.1.20 \
local 192.168.1.10 \
dev enp3s0
ip link set vxlan100 up
# Attach to bridge
ip link set vxlan100 master br0
# Multicast mode (requires multicast routing)
ip link add vxlan200 type vxlan \
id 200 \
dstport 4789 \
group 239.1.1.1 \
dev enp3s0
# Add static FDB entry for unicast mode
bridge fdb add 00:11:22:33:44:55 dev vxlan100 dst 192.168.1.30
# Check NIC VXLAN offload support
ethtool -k enp3s0 | grep -i vxlan
# Verify MTU headroom
ip link show vxlan100
ip link set vxlan100 mtu 1450macvlan and ipvlan: Multiple IPs on One Physical NIC
macvlan creates virtual interfaces that each have their own MAC address, all sharing a single physical NIC. The physical NIC must be in promiscuous mode to receive frames for all the virtual MACs. Four modes exist: private, vepa, bridge, and passthru. The `bridge` mode is most common - macvlan interfaces on the same parent can communicate directly without going through an external switch.
ipvlan is the L3 equivalent: virtual interfaces share the parent's MAC but have distinct IPs. The kernel routes between them at L3. This avoids the promiscuous mode requirement and works better on switches with strict MAC security. ipvlan has two sub-modes: L2 (Ethernet forwarding) and L3 (IP routing only, no ARP between slaves).
For container networking without a bridge, macvlan in bridge mode is faster than a veth-plus-bridge setup because it eliminates the bridge FDB lookup. Docker's macvlan network driver uses this. The tradeoff: containers cannot reach the host's IP on the parent interface, because the physical NIC does not receive its own macvlan children's traffic when in bridge mode. Work around this by creating a macvlan interface on the host itself for the same parent.
In our throughput tests between a macvlan container and an external host, we measured 9.4Gbps on a 10G NIC versus 8.9Gbps through a Linux bridge - a 5.6% improvement. For latency-sensitive workloads processing thousands of small packets per second, macvlan's lower CPU overhead is meaningful.
# Create macvlan interface (bridge mode)
ip link add macvlan0 link enp3s0 type macvlan mode bridge
ip link set macvlan0 up
ip addr add 192.168.1.50/24 dev macvlan0
# Host macvlan to reach containers from host
ip link add macvlan-host link enp3s0 type macvlan mode bridge
ip link set macvlan-host up
ip addr add 192.168.1.51/24 dev macvlan-host
# ipvlan - L3 mode
ip link add ipvlan0 link enp3s0 type ipvlan mode l3
ip link set ipvlan0 up
ip addr add 192.168.1.60/24 dev ipvlan0
# Move macvlan into a network namespace (container pattern)
ip netns add container1
ip link add macvlan-c1 link enp3s0 type macvlan mode bridge
ip link set macvlan-c1 netns container1
ip netns exec container1 ip link set macvlan-c1 up
ip netns exec container1 ip addr add 192.168.1.70/24 dev macvlan-c1Persisting Virtual Network Configuration
All `ip` commands are runtime-only. A reboot wipes every virtual interface, route, and address set this way. Persisting config depends on your distro and toolchain.
On Ubuntu 24.04, netplan manages interfaces. Write YAML to `/etc/netplan/` and run `netplan apply`. Netplan generates systemd-networkd or NetworkManager config depending on the renderer. For bridges and VLANs, use `renderer: networkd` - NetworkManager's bridge support has historically been less reliable in headless server setups.
On RHEL 9 / Alma 9 / Rocky 9, NetworkManager is the default. Use `nmcli` to create bridges and VLANs, which writes config to `/etc/NetworkManager/system-connections/`. Alternatively, write raw `[connection]` INI files directly.
For systems you manage with Ansible, the `community.general.nmcli` module handles most virtual interface types. For systemd-networkd directly, write `.netdev` files for virtual devices and `.network` files for addressing. This approach is distribution-agnostic and works well in immutable OS setups like Fedora CoreOS.
One production pattern we use: create all virtual interfaces in a systemd service that runs `ip` commands from a shell script, ordered after `network-pre.target` and before `network.target`. It is less elegant than netplan but survives distribution upgrades without config migration.
# /etc/netplan/10-bridge.yaml (Ubuntu 24.04)
network:
version: 2
renderer: networkd
ethernets:
enp3s0:
dhcp4: false
bridges:
br0:
interfaces: [enp3s0]
addresses: [192.168.10.1/24]
routes:
- to: default
via: 192.168.10.254
parameters:
stp: false
forward-delay: 0
nameservers:
addresses: [1.1.1.1, 8.8.8.8]
vlans:
enp3s0.100:
id: 100
link: enp3s0
addresses: [10.0.100.1/24]Debugging Virtual Network Issues
When traffic disappears in a virtual network, start at L2 with `bridge fdb show` and work up. The most common causes on Linux are: ARP not reaching the right interface, netfilter dropping forwarded frames, MTU mismatches in VXLAN setups, and IP forwarding disabled.
The `tc` tool (traffic control) is the fastest way to capture traffic on any virtual interface without needing a mirror port. Attach a `mirred` action with a `clsact` qdisc to copy packets to a dummy interface, then run tcpdump on it. Or just run tcpdump directly on the virtual interface - it works on bridges, vlans, tun, and veth.
`ss -s` shows socket statistics including UDP socket counts - useful for verifying VXLAN traffic is arriving. `nstat -az` dumps all kernel network counters; filter for drop counters: `nstat -az | grep -i drop`.
For bridge-specific issues, `bridge monitor` watches for real-time FDB changes, topology changes, and MDB (multicast) events. Running it during a connectivity test shows exactly which MAC addresses are being learned and from which ports.
# Check IP forwarding
sysctl net.ipv4.ip_forward
sysctl net.ipv6.conf.all.forwarding
# Watch bridge FDB changes in real time
bridge monitor fdb
# Capture on bridge interface
tcpdump -i br0 -nn -e arp
# Check for drops across all virtual interfaces
nstat -az | grep -iE 'drop|discard|error'
# Verify MTU through the stack
ping -M do -s 1472 -c3 10.0.100.2 # 1472 + 28 = 1500 byte frame
# Check iptables FORWARD chain (common silent drop source)
iptables -nvL FORWARD
# Trace a packet through netfilter
modprobe nf_log_ipv4
iptables -t raw -A PREROUTING -s 10.8.0.0/24 -j TRACE
# Read trace output
journalctl -k | grep 'TRACE:'
# Remove trace rule when done
iptables -t raw -D PREROUTING -s 10.8.0.0/24 -j TRACE