VRRP and keepalived for Gateway/Service Failover

Introduction¶

VRRP (Virtual Router Redundancy Protocol) lets a group of nodes present a single virtual IP (VIP) and MAC. One node ("master") owns the VIP; if it fails, another takes over in ~1–3 seconds. Clients keep using the same address. keepalived is the standard Linux daemon.

How VRRP works¶

• Nodes in a VRRP instance share a virtual router ID (VRID, 1–255) and a priority (1–254).
• The highest-priority node becomes master, claims the VIP, and sends advertisements (multicast 224.0.0.18, protocol 112) every advert_int (default 1s).
• Backups listen. If they miss ~3 adverts (3 × advert_int + skew), the highest-priority backup promotes itself, claims the VIP, and sends a gratuitous ARP so switches relearn the MAC.
• The virtual MAC is 00:00:5e:00:01:<VRID> — so failover doesn't even change the MAC clients see (no ARP cache issue on clients, only on switches).

preempt (default on): a recovered higher-priority node takes master back. nopreempt leaves the current master alone to avoid a second blip.

keepalived: a floating VIP (active/passive)¶

# /etc/keepalived/keepalived.conf  — NODE A (higher priority)
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass changeme
    }
    virtual_ipaddress {
        10.0.0.10/24 dev eth0
    }
}

# NODE B — lower priority, state BACKUP, priority 100, everything else identical

state MASTER vs BACKUP is just the initial state — priority decides who actually wins.

Add a health check¶

A VIP that stays on a node whose service is dead is useless. Use a vrrp_script that lowers priority (or fails the instance) when the service is unhealthy:

vrrp_script chk_nginx {
    script "/usr/bin/killall -0 nginx"   # or a curl to /healthz
    interval 2
    weight -60          # drop priority by 60 if the check fails
    fall 2
    rise 2
}

vrrp_instance VI_1 {
    ...
    track_script {
        chk_nginx
    }
}

Now if nginx dies on the master, its effective priority (150 − 60 = 90) drops below the backup's 100, and the VIP moves.

keepalived also does LVS¶

virtual_server blocks turn keepalived into an L4 load balancer (IPVS) with real-server health checks — an active/passive pair of keepalived nodes each running LVS, VIP floated by VRRP. Common for a DIY load balancer.

Verification and troubleshooting¶

ip addr show eth0 | grep 10.0.0.10        # who has the VIP?
journalctl -u keepalived -f               # "Entering MASTER STATE" / "BACKUP STATE"
tcpdump -i eth0 -n vrrp                    # see advertisements + priority

• Both nodes think they're master (split brain) — VRRP adverts not reaching each other: a switch blocking multicast/IGMP snooping dropping 224.0.0.18, protocol 112 filtered by a firewall, different virtual_router_id, or auth_pass mismatch. Both hold the VIP → duplicate IP, chaos. Fix the adverts; consider unicast_peer instead of multicast on hostile networks.
• VIP doesn't move on failure — no health check (keepalived only fails over on node/keepalived death, not service death). Add a vrrp_script.
• Failover works but clients hang ~30s — the gratuitous ARP didn't propagate (switch port security, or the client cached the ARP). keepalived sends GARP by default; check garp_master_delay. Some clouds don't allow GARP-based VIP moves — use their API (notify_master script calling the cloud CLI to reassign the IP).
• Constant flapping — weight math puts the two nodes' priorities too close; a marginal health check toggling. Widen the gap, add fall/rise hysteresis.
• preempt caused a second outage — a flapping master keeps reclaiming the VIP. Set nopreempt on the backup (and state BACKUP on both).
• Works on-prem, not in AWS/Azure — cloud networks don't do L2 GARP/VRRP for arbitrary IPs. Use a secondary private IP + notify scripts that call the cloud API, or the cloud's own load balancer / floating IP.
• VRRPv3 vs v2 — v3 supports IPv6 and sub-second timers; both ends must match version.

Related tools and reading¶

• On-site: Ping test, Port Checker.
• Related posts: HAProxy basics, ARP and the neighbour table.