How LACP Actually Works (And How To Configure It On Debian)

Everybody’s first instinct when they need more bandwidth is to ask for two ports. Here’s what happens next: they configure the server, the switch does something different, and they spend an evening wondering why a 2×10Gb bond delivers the same throughput as a single 10Gb link.

That outcome isn’t bad luck. It’s the predictable result of a protocol that almost nobody reads the spec for.

This post covers what LACP is actually negotiating on the wire, why aggregation width is usually narrower than people expect, and how to configure and, more importantly, verify a bond on Debian.

What LACP is, precisely

LACP is Link Aggregation Control Protocol, standardised as IEEE 802.1AX (which obsoleted 802.3ad; you’ll still see the older number everywhere, including in the Linux driver, which still calls the mode 802.3ad).

It exists to solve one specific problem: getting multiple physical links between two devices to behave as one logical link, without static configuration.

That’s the entire point. LACP is a negotiation protocol. Static EtherChannel, the older vendor-specific approach where you manually tell the switch which ports are in a group, does the same grouping but with no agreement mechanism.

LACPDUs are the packets doing the negotiating. They go to the reserved multicast address 01:80:C2:00:02 and are never forwarded. Each LACPDU carries:

Ports only move traffic once both sides independently agree they’re in the same aggregator.

The failure mode this prevents

This is the part worth internalising, because it explains why “just use static EtherChannel” is bad advice.

Configure static EtherChannel on the switch (LACP disabled) and LACP on the host, and you get a forwarding loop:

1. Switch sees both ports, immediately puts them in the static LAG.
2. Host’s LACP state machine sees no LACP replies (the switch isn’t running LACP), so it keeps both ports out of its aggregator.
3. Host transmits out ens18. Switch forwards to its other LAG member, ens19.
4. Frame arrives back at the host on ens19, which the host is happy to receive.

Broadcast storm. Same failure if you get the pairing wrong: two host ports to two ports on the same switch in the same LAG, where the switch can’t tell which is which.

With LACP on both sides, the mismatched port simply stays in a DOWN / not-selecting state and nothing loops. The port fails closed instead of failing catastrophically.

Never configure LACP on one side only. If you’re unsure what the switch is doing, check before you touch the host.

Static EtherChannel vs LACP, side by side

The loop above is the dramatic reason to prefer LACP. It’s not the only difference, and it isn’t even the biggest operational one.

The row people underrate is failure detection. A static bundle has no heartbeat whatsoever. The switch knows a port went down because the physical link dropped, which is fine, but it cannot tell the difference between “this cable is gone” and “the far end is wedged, looping packets into a black hole, and the LED is still lit.” Neither can the host. LACP exists precisely to answer that question.

The static configuration, so you know what it looks like

This is what balance-rr looks like on the host side:

auto bond0
iface bond0 inet static
    bond-slaves ens18 ens19
    bond-mode 0
    bond-miimon 100
    address 10.20.30.40/24
    gateway 10.20.30.1

Compare it to the LACP version in the Debian section below: one line differs. bond-mode 0 instead of bond-mode 802.3ad, and no bond-lacp-rate. That is genuinely all that differs on the host, which is precisely why the misconfiguration happens so easily. The switch side is what actually has to agree, and it’s also what people are slower to check.

Two properties of balance-rr worth naming:

• It is round-robin per packet, not per flow. TCP requires ordered delivery within a connection. If consecutive packets of one session leave on different slaves, they can arrive out of order, and the host’s own stack does not reassemble them; the far end sees the damage. This is why round-robin is unsuitable for general TCP/IP traffic, and why mode 0 is not “the fast one.”
• It requires the switch side to be a static bundle. Pair it with LACP on the switch and you land back in the loop above, with the added problem that the host is happily sending from both ports while the switch refuses to aggregate them.

When static EtherChannel is genuinely the right answer

I’d rather say this than let the post read as “static is always wrong.”

• The switch doesn’t implement LACP. Older gear, some SMB NAS and SAN arrays, a handful of appliances and managed switches with a trimmed feature set. Static is a one-way declaration that both ends can honour with zero protocol.
• Point-to-point links you fully control, where both ends are yours and the pairing is documented and physical. There’s no third party to misconfigure, so the loop risk is a wiring problem you can simply not make.
• Deliberate deterministic distribution. Some storage appliances and backup appliances expect traffic on a specific member, and static lets you force it. LACP’s hashing is designed to spread flows, which is the opposite of what those devices want.
• Legacy pairs during a migration window, where one end genuinely can’t be changed.

If it’s a server on a modern datacenter switch, none of these apply.

Why 2×10Gb does not give you 20Gb

This is the part that generates most of the disappointment.

LACP does not split one connection across links. It hashes flows onto links, and then each flow rides a single physical link for its entire lifetime. No striping. No reassembly. No packet-level parallelism.

The kernel’s default policy is layer2: source MAC XOR destination MAC XOR EtherType, modulo slave count. For a server talking to the world through a single default gateway, the source MAC and destination MAC are usually the same values for every packet. Most of your traffic hashes to one slave and stays there.

Set xmit_hash_policy=layer3+4 and you hash on source/destination IP and port. Now distinct TCP sessions spread across links. Better, but still bounded.

The ceiling is your single largest flow. A 40Gb/s storage mount or one bulk transfer will run at link speed regardless of how many links you added. Aggregation increases aggregate capacity across many flows. It does not make any one flow faster.

Before you buy switch ports, measure your actual distribution:

# Where is traffic going, and how much of it is one conversation?
ss -tn state established | awk 'NR>1 {print $4, $5}'

If you have a handful of flows dominating, you need more bandwidth per link, not more links.

Choosing a mode

Linux ships the bonding driver with eight modes. Three matter in practice.

Modes 5 (balance-tlb) and 6 (balance-alb) do adaptive load balancing and require no switch support, useful when you’re stuck with dumb or unmanaged switches. They work by per-host ARP negotiation, which has its own set of oddities. Know they exist; don’t reach for them first.

If your switch supports LACP, use 802.3ad.

Configuring on Debian

Debian still ships ifupdown (/etc/network/interfaces) as the default on servers. That’s what I’ll use here. If you’re on systemd-networkd, the equivalent config is at the bottom.

Install

sudo apt update
sudo apt install -y ifupdown ethtool
modprobe bonding
lsmod | grep bonding

The switch side first

Do this before touching the host. Cisco-style syntax:

interface range GigabitEthernet1/0/1-2
 channel-group 1 mode active
 lacp system-priority 100

Generic equivalent: create a port-channel / link-aggregate of the two ports, set its mode to LACP active or LACP passive. Vendor wording varies but the setting is the same.

If the switch runs RSTP/STP on those ports, mark the port-channel as an edge port. A bond is a single logical segment to STP; a BPDU on one member flapping can otherwise block the whole bundle.

The host side

# /etc/network/interfaces
auto bond0
iface bond0 inet static
    address 10.20.0.10/24
    gateway 10.20.0.1

    bond-slaves            ens18 ens19
    bond-mode              802.3ad
    bond-xmit-hash-policy  layer3+4

    # Link monitoring: do not skip this. Without it, a silent
    # cable/optic failure goes undetected until the next TCP timeout.
    bond-miimon            100
    bond-updelay           200
    bond-downdelay         200

    # Ask the partner for 1-second LACPDUs instead of 30-second.
    bond-lacp-rate         fast
    bond-lacp-active       1

    # Don't assert carrier until at least one link is genuinely up.
    bond-min-links         1

    # Drop frames arriving on non-active members rather than
    # delivering duplicates up the stack.
    bond-all-slaves-active 0

Then bring it up without rebooting, if you can:

sudo systemctl restart networking

Remote-host warning: restart networking drops the bond, and if anything is wrong you lose your session. Have out-of-band console or IPMI access before you touch a remote machine. This is the single most common way this goes badly.

bond-updelay and bond-downdelay must be multiples of bond-miimon; otherwise the kernel rounds them down, silently. 200 and 100 is fine.

systemd-networkd equivalent

# /etc/systemd/network/20-bond0.network
[Match]
Name=ens18 ens19

[Network]
Bond=bond0

[NetworkBond]
Mode=802.3ad
TransmitHashPolicy=layer3+4
LACPTransmitRate=fast
MIIMonitoringSec=1s
UpDelaySec=2s
DownDelaySec=2s
AllSlavesActive=false

# /etc/systemd/network/20-bond0.netdev
[NetDev]
Name=bond0
Kind=bond

sudo systemctl restart systemd-networkd

Verifying it: the part people skip

Do not trust that it came up. Check that it’s actually aggregating.

# Is the bond up, and in which mode?
cat /proc/net/bonding/bond0

You want to see:

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200

802.3ad info
LACP rate: fast
Min links: 1
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
    Aggregator ID: 1
    Number of ports: 2        <-- both links, not one
    Actor Key: 9
    Partner Key: 1
    Partner Mac Address: aa:bb:cc:dd:ee:ff

Number of ports: 2 is the line that matters. If it reads 1, you are not aggregating: one link failed to negotiate, or the switch side isn’t configured.

Two more checks:

# What has the kernel negotiated as the hash policy?
ip -d link show bond0 | grep -o 'xmit_hash_policy[^ ]*'

# Confirm both slaves are actually enslaved and up
ip -br link show | grep -E 'ens18|ens19|bond0'

You can also watch LACPDUs on the wire to confirm the exchange is really happening:

sudo tcpdump -i ens18 -e -n ether proto 0x8809

0x8809 is the LACP ethertype. If you see LACPDUs arriving, the peer is talking LACP. If you see nothing, your partner isn’t.

Prove the failover actually works

An untested failover is a guess. When you get a maintenance window, not during an incident, pull one link and watch:

# Watch bond state transitions live
watch -n1 "grep -E 'MII Status|MII Polling' /proc/net/bonding/bond0"

# Confirm the aggregator drops to one port
watch -n1 "grep -A3 'Active Aggregator Info' /proc/net/bonding/bond0"

Then pull the optic, or ip link set ens19 down in a test window. The aggregator should fall to Number of ports: 1 and traffic should continue uninterrupted.

Now think about the timing, because this is where SLAs get quietly missed.

LACP declares a port dead after missing 3 consecutive LACPDUs:

The default is slow. Ninety seconds of dropped traffic is fine on a datacenter fabric where your application retries quickly. It is not fine for a credit-card authorization path, a VoIP call, or anything holding a TCP session open.

That’s why the config above sets bond-lacp-rate fast. lacp_rate is a request: it asks the partner to transmit faster. Both sides need to agree, so set it on the switch too.

Note this only covers link-level failure. If the switch dies but the host’s links stay up, the host has no idea; LACP is not an end-to-end health check. For that you need arp_interval / arp_ip_target monitoring, or an external active probe.

Tuning notes

min_links. Only affects 802.3ad. Setting it above zero means the bond won’t assert carrier until N links are up. Useful before a maintenance operation, so services don’t start on a single link you thought was redundant.

ad_select. Which aggregator wins if you have several. stable (default) keeps the current one until it fully fails. bandwidth and count reselect on any change, trading a little flap risk for better resilience on partial failure.

all_slaves_active. Leave at 0. Setting it to 1 delivers duplicate frames arriving on standby members straight up the stack. That’s a debugging aid, not a production setting.

xmit_hash_policy choice. For general server traffic, layer3+4. It’s usually the right default. The catch: if traffic is encapsulated (VXLAN, GRE, IP-in-IP), you want the encap3+4 variant, or the hash ignores the inner tuple and everything lands on one link again.

rp_filter. If you see asymmetric return traffic after enabling a bond, check strict reverse-path filtering; it will happily drop replies arriving on a different slave than the request left on:

sysctl net.ipv4.conf.all.rp_filter net.ipv4.conf.bond0.rp_filter
# 2 = loose, which is what you generally want on a bond

The short version

LACP negotiates agreement; static EtherChannel assumes it, and gets a loop when it’s wrong. Aggregation hashes flows rather than striping them, so your ceiling is your single largest flow, not your link count. bond-mode 802.3ad with xmit_hash_policy layer3+4 is the default you want. Set lacp_rate fast if 90 seconds of undetected failure would breach your SLA. And always read Number of ports out of /proc/net/bonding/bond0; a bond that came up as one link still looks like it came up.

Configure the switch side first. Test the failover on purpose, in a window, not during an incident.