Tuning Open File Limits on Linux

Introduction¶

Too many open files (EMFILE) on a busy web server, database, or proxy is almost always a limit that's set too low for the workload — not a leak. The confusion is that there are four separate limits and raising the wrong one does nothing.

The four limits¶

Also relevant: net.core.somaxconn (listen backlog) and per-user process counts — different limits, similar symptoms.

Check what a running process actually has¶

# The limits the process is running under
cat /proc/<pid>/limits | grep "open files"

# How many it currently has open
ls /proc/<pid>/fd | wc -l

# System-wide
cat /proc/sys/fs/file-nr        # allocated  unused  max
sysctl fs.file-max fs.nr_open

If /proc/<pid>/limits says 1024 and the process needs 50k, that's your limit — regardless of fs.file-max.

Raising it — the right layer¶

For a systemd service (the common case)¶

sudo systemctl edit myapp.service

[Service]
LimitNOFILE=262144

sudo systemctl daemon-reload
sudo systemctl restart myapp.service
cat /proc/$(pgrep -f myapp)/limits | grep "open files"

systemd services do NOT read /etc/security/limits.conf — that's PAM, and systemd doesn't go through PAM for services. This trips people up constantly.

For interactive / login sessions¶

/etc/security/limits.d/90-nofile.conf:

*        soft nofile 65536
*        hard nofile 262144
root     soft nofile 65536
root     hard nofile 262144

Requires pam_limits.so in the PAM stack (default) and a fresh login. Verify with ulimit -n (soft) and ulimit -Hn (hard).

System-wide ceiling¶

Only if fs.file-max is genuinely the bottleneck (millions of fds across all processes):

# /etc/sysctl.d/99-files.conf
fs.file-max = 2097152
fs.nr_open  = 1048576      # raise if you want a per-process ulimit above ~1M

sudo sysctl --system

Verification and troubleshooting¶

# Reproduce and watch fd count climb
watch -n1 'ls /proc/$(pgrep -f myapp)/fd | wc -l'

• Raised limits.conf but the service still hits 1024 — it's a systemd service; use LimitNOFILE= in a drop-in, not PAM limits.
• Set LimitNOFILE= but /proc/<pid>/limits unchanged — no daemon-reload, or the service was reloaded (not restarted — limits apply at process start), or a wrapper/su in ExecStart reset them.
• fd count keeps climbing without bound — a genuine leak. ls -l /proc/<pid>/fd and look for hundreds of the same socket or file. lsof -p <pid> | sort -k9 | uniq -c -f8 | sort -rn.
• ulimit -n unlimited refused — you can't; there's a hard cap at fs.nr_open. Raise that sysctl if you truly need >1M.
• Container hits the limit — the container's limits come from the runtime (docker run --ulimit nofile=65536:65536, or LimitNOFILE on the containerd/docker service, or the Kubernetes node). The image's own limits.conf is irrelevant for the main process.
• accept: too many open files but fd count is well under the limit — could be fs.file-max (system total) reached by other processes, or you're out of ephemeral ports / conntrack entries, which present similarly.

Related tools and reading¶

• On-site: Linux commands reference, Port Checker.
• Related posts: Finding what is using a port, Understanding Linux load average.