Performance Tuning Linux Servers for Web Workloads

A Linux server that feels slow is not automatically a server that needs a larger worker_connections value or a new block of sysctl settings. It may be CPU saturation, storage latency, a full connection queue, an application pool that is too small, or an upstream dependency. Performance tuning is therefore a measurement loop: define the workload, capture a baseline, change one variable, measure again, and keep a tested rollback.

Start with a baseline, not a recipe

Record requests per second, p50/p95/p99 latency, HTTP error rate, and the exact test URL or transaction. During a representative period, collect both utilization and saturation:

uptime
free -m
vmstat 1
mpstat -P ALL 1
iostat -xz 1
pidstat 1
ss -s

uptime load averages need context; they can include tasks waiting for I/O, not just runnable CPU work. vmstat exposes run-queue, swap, and blocked-task behavior, while iostat -xz helps distinguish a busy disk from an idle one. Keep the same test duration, concurrency, cache state, and client location for before-and-after comparisons. A benchmark run from the server itself can measure local loopback behavior instead of the network path users experience.

For deeper investigations, the USE method is a useful discipline: check utilization, saturation, and errors for each resource. Brendan Gregg’s active benchmarking guidance is a good reminder that a short test can benchmark one component while you think you measured the whole service.

Nginx: match capacity to the workload

Nginx documents worker_processes auto as a sensible starting point because it attempts to detect available CPU cores. It is not a promise that one worker per core is optimal: TLS, compression, upstream waits, and CPU affinity can change the result. worker_connections defaults to 512 and counts upstream connections too, not only browser sockets. Its effective ceiling is also constrained by the worker’s file-descriptor limit. Our earlier file-descriptor and Nginx worker-connections deep dive covers that limit relationship in detail.

A small, testable starting point might look like this:

worker_processes auto;
worker_rlimit_nofile 65535;

events {
    worker_connections 2048;
}

http {
    sendfile on;
    keepalive_timeout 30s;
    keepalive_requests 1000;
}

These are examples, not universal targets. keepalive_timeout defaults to 75 seconds and keepalive_requests to 100 in current Nginx documentation; shortening or extending them changes memory, connection reuse, and latency trade-offs. A reverse proxy also needs upstream keepalive configured consistently, while a static file server may benefit more from correct cache headers and page-cache behavior than from more workers. Use nginx -t before a reload, and compare connection counts and tail latency after the change. sendfile on can reduce copies for static files, but filters that transform content, such as compression, can prevent the zero-copy path from applying.

Queues and file descriptors

Read the current values before changing them:

sysctl net.core.somaxconn net.ipv4.tcp_max_syn_backlog
ulimit -Sn
ulimit -Hn
sysctl fs.file-max fs.nr_open

Modern kernels document net.core.somaxconn as 4096 (it was 128 before Linux 5.4), but the value only matters when the application and listen backlog use it. If you intentionally set an Nginx listen ... backlog=, align the two values and inspect queue behavior under load. tcp_max_syn_backlog covers partially established connections; raising it is a response to observed pressure, not a substitute for fixing a slow application.

Per-process limits, systemd’s LimitNOFILE, fs.nr_open, and the system-wide fs.file-max are different controls. Raising one while another remains lower does not create capacity. The systemd LimitNOFILE documentation and kernel filesystem sysctl documentation explain the boundaries. Treat old recipes for tcp_tw_reuse, syncookies, and huge fs.file-max values with suspicion: current kernel semantics and defaults matter, and syncookies are a SYN-flood fallback rather than a legitimate-traffic scaling switch.

Memory, storage, and application pools

The kernel’s default vm.swappiness is 60 on documented modern Linux systems, with a range of 0–200. A value above 100 is specifically described as potentially useful for faster in-memory swap such as zram or zswap; it is not a generic “make servers faster” setting. vm.vfs_cache_pressure defaults to 100. Lowering it makes the kernel prefer retaining dentry and inode caches, which can help metadata-heavy workloads but can also compete with application memory. Measure reclaim, major faults, and application RSS before touching either setting. The kernel VM documentation is the authority for the current behavior.

For PHP-FPM, pm.max_children is the cap on concurrent requests. Size it from measured average child RSS and memory actually available after the OS, database, and cache budgets—not from a copied formula. The heuristic available application memory / average child RSS is only a starting estimate; leave headroom and verify queueing, latency, and swap. pm.max_requests defaults to unlimited and can be set to recycle children when a library leaks memory. request_terminate_timeout can protect the pool from pathological requests. Node, Python, Gunicorn, and other runtimes have analogous worker or thread pools: identify the pool bottleneck before increasing Nginx capacity.

Caching has several layers with different failure modes: filesystem page cache, Nginx proxy or FastCGI cache, HTTP Cache-Control headers, and application opcode caches. Invalidate each deliberately. A cache hit-rate increase that serves stale or personalized content is not a performance success.

Validate, monitor, and roll back

Run wrk, hey, or a comparable client from a separate machine, for a steady period long enough to reach a stable state. Watch vmstat, mpstat, iostat, ss -s, Nginx access logs, and application metrics while the test is running. Compare percentiles and error rates, not only average latency. For ongoing visibility, retain sysstat history and expose Nginx or PHP-FPM status endpoints only to trusted monitoring addresses; these endpoints reveal operational details.

Runtime sysctl -w changes disappear at reboot. If a change helps, record it in a dedicated /etc/sysctl.d/99-web-tuning.conf, apply it with sysctl --system, and keep the file in version control. For a systemd override, use systemctl daemon-reload, validate the unit, and restart only when required. Roll back by restoring the prior value or removing the drop-in, then re-run the same baseline test. Never use echo 1 > /proc/sys/vm/drop_caches as a tuning fix; it is a diagnostic action that deliberately discards useful cache state.

The right setting depends on whether the machine serves static assets, terminates TLS, proxies to an application, or also hosts a database. Measure the bottleneck, make the smallest defensible change, validate it under the real workload, and keep the old configuration one command away.

Sources and further reading

• Nginx core module documentation
• Nginx HTTP core module documentation
• Nginx upstream keepalive documentation
• F5: Tuning NGINX for Performance
• Linux kernel VM sysctl documentation
• Linux kernel network sysctl documentation
• PHP-FPM configuration manual
• Netflix: Linux Performance Analysis in 60,000 Milliseconds