sysctl Network Tuning for Servers

Introduction¶

Default kernel network settings are conservative. On a high-connection-rate server (reverse proxy, API gateway, load balancer) a handful of sysctl changes help. Change one thing at a time and measure — blind tuning often hurts.

Where to put them¶

# /etc/sysctl.d/99-network-tuning.conf
...settings...

sudo sysctl --system          # apply
sysctl net.core.somaxconn     # verify a value

The settings that matter¶

Accept queue / backlog¶

net.core.somaxconn = 4096
net.ipv4.tcp_max_syn_backlog = 8192
net.core.netdev_max_backlog = 16384

somaxconn caps the completed-connection queue; the app also has to pass a matching backlog to listen() (nginx listen ... backlog=4096;, or the app's config). Symptom of it being too low: ss -tlnp shows Recv-Q pegged at Send-Q on the listen socket, and clients see resets/timeouts under load.

Ephemeral ports and TIME_WAIT (for outbound-heavy servers)¶

A proxy makes many outbound connections to backends:

net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_reuse = 1        # reuse TIME_WAIT sockets for new OUTBOUND connections (safe)
net.ipv4.tcp_fin_timeout = 15

Do not set tcp_tw_recycle — it's removed in modern kernels and broke NAT'd clients. tcp_tw_reuse=1 is the safe one and only affects outbound.

Better fix: connection pooling / keep-alive to backends so you're not opening a connection per request at all.

Buffers (only if you have bandwidth-delay-product need)¶

net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 131072 16777216
net.ipv4.tcp_wmem = 4096 16384 16777216

Helps high-throughput transfers over high-latency links (cross-region replication, big uploads). Useless for lots of small requests, and oversized buffers waste memory (multiply by connection count).

SYN flood protection¶

net.ipv4.tcp_syncookies = 1        # usually already on

Leave on. Under a real SYN flood it lets legitimate connections through by not allocating state until the ACK.

Keepalives (detect dead peers faster)¶

net.ipv4.tcp_keepalive_time = 120
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 3

Only affects sockets that enable SO_KEEPALIVE. Helps clean up half-open connections from clients that vanished (mobile, NAT timeouts).

conntrack (if the box does NAT / stateful firewall)¶

net.netfilter.nf_conntrack_max = 524288
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30

See ss and conntrack for connection debugging. On a box with no NAT/stateful rules, conntrack may not even be loaded — good.

BBR congestion control (egress-heavy servers)¶

net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

BBR often improves throughput on lossy/long paths (CDN origin, video). Test — it's not universally better.

What NOT to blindly change¶

• tcp_timestamps — leave on (needed for PAWS and tw_reuse).
• tcp_sack — leave on.
• Disabling tcp_slow_start_after_idle can help keep-alive-heavy servers, minor.
• Anything you saw in a 2012 blog post — kernels changed.

Verification and troubleshooting¶

# Before/after
ss -s                                  # socket summary by state
ss -tln 'sport = :443' | head          # Recv-Q/Send-Q on the listener
nstat -az | grep -Ei 'TcpExtListenDrops|TcpExtListenOverflows|SyncookiesSent'
cat /proc/net/sockstat
sysctl net.ipv4.tcp_congestion_control

• ListenOverflows / ListenDrops climbing — accept backlog too small (kernel and app), or the app isn't accepting fast enough (add workers).
• Cannot assign requested address on outbound — ephemeral port exhaustion. Widen ip_local_port_range, tcp_tw_reuse=1, and pool connections.
• Buffers raised, no change / more memory used — the workload isn't bandwidth-limited. Revert.
• nf_conntrack: table full — raise nf_conntrack_max, shorten timeouts, or NOTRACK non-stateful traffic.
• Changes didn't persist — file not in /etc/sysctl.d/ with .conf, or sysctl --system not run, or a later-sorting file overrides it.
• sysctl: cannot stat /proc/sys/net/netfilter/... — the module isn't loaded; the setting only exists once conntrack is in use.

Related tools and reading¶

• On-site: Port Checker, Subnet Calculator.
• Related posts: ss and conntrack for connection debugging, TCP congestion control basics.