About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.
When a Linux shell prints fork: retry: Resource temporarily unavailable, it asked the kernel for a new process and the kernel answered EAGAIN. The fork(2) manual lists four limits under its first EAGAIN entry: the per-user process limit, the system-wide thread limit, the maximum process ID, and the cgroup process limit. Memory shortage is a different error. Find which limit was hit before you raise any of them.
What the error means
The text after the colon is the C library's text for EAGAIN. The errno(3) page gives it as "Resource temporarily unavailable". The fork: retry prefix comes from bash. In its source, bash calls fork(), and while it fails with EAGAIN bash reports fork: retry, sleeps, and tries again. The first sleep is one second and the sleep doubles after each failure, up to a maximum. This is read from the bash source, not from the bash manual, and the mirror's version is not shown. Other shells word the error differently.
The fork(2) page says EAGAIN means "A system-imposed limit on the number of threads was encountered", and lists:
- the
RLIMIT_NPROCsoft resource limit, which limits the number of processes and threads for a real user ID; - the kernel's system-wide limit
/proc/sys/kernel/threads-max; - the maximum number of PIDs,
/proc/sys/kernel/pid_max; - the PID limit
pids.maximposed by the cgroup process number (PIDs) controller.
A different error covers memory: fork(2) says ENOMEM means fork "failed to allocate the necessary kernel structures because memory is tight". So a server with free memory can still refuse to fork. The kernel's cgroup v2 page makes the same point: "a fork bomb is likely to exhaust the number of tasks before hitting memory restrictions".
Two details matter when you read the numbers. The getrlimit(2) page says RLIMIT_NPROC counts processes "or, more precisely on Linux, threads" for the real user ID, so a service with many threads uses the limit up quickly. It also says the limit "is not enforced for processes that have either the CAP_SYS_ADMIN or the CAP_SYS_RESOURCE capability, or run with real user ID 0". The cgroup v2 page says PIDs in the controller "refer to TIDs, process IDs as used by the kernel", so threads count there too.
Likely causes
| Cause | What you see | How to confirm |
|---|---|---|
Per-user limit (RLIMIT_NPROC, LimitNPROC=, nproc in limits.conf) |
One non-root account cannot start anything; root can | prlimit --pid PID for a process of that account, and a count of that account's threads |
systemd TasksMax= for one service (cgroup pids.max) |
The one service fails to start workers; the rest of the host is fine | systemctl status shows its task count; systemctl show shows the limit; cgroup pids.events counts hits |
| Runaway process creation (a loop, a leak, a fork bomb) | Task count climbs until a limit stops it | ps sorted by thread count |
System-wide threads-max or pid_max |
Every account is affected, including root | Compare total threads with the two files |
Prove the cause before changing anything
All of these are read-only.
1. Which account or service is full? List processes with their thread counts, largest first:
ps -eo pid,user,nlwp,args --sort=-nlwp
The ps page documents -e (all processes), -o (the output columns), nlwp ("number of lwps (threads) in the process") and --sort. A process or account with a very large total points at the cause.
2. The per-user limit. For a process that belongs to the failing account:
prlimit --pid 13134
The prlimit page documents -p, --pid and gives this form as "Display limit values for all current resources". Replace the example process ID with a real one and read the line for the maximum number of processes (the page lists -u, --nproc as "Maximum number of processes"). To count the threads that account already runs, list them with ps -L -U appuser -o pid,lwp (the ps page documents -L as "Show threads, possibly with LWP and NLWP columns" and -U as selecting by real user ID or name, which is what RLIMIT_NPROC counts). Each line below the header is one thread. Compare that count with the soft limit.
3. The service limit under systemd. If the failing work is a service:
systemctl status app.service
systemctl show --property=TasksMax --property=EffectiveTasksMax app.service
The systemd.resource-control page says TasksMax= controls the pids.max control group attribute and that "The effective configuration is reported as EffectiveTasksMax=". Its default comes from DefaultTasksMax= in systemd-system.conf, which, on the systemd-system.conf page, "Defaults to 15% of the minimum of kernel.pid_max=, kernel.threads-max= and root cgroup pids.max"; that page gives 4915 as the value with the default kernel.pid_max. The systemctl status example on the systemctl page includes a Tasks: line.
4. The cgroup counters. The kernel cgroup v2 page defines these files for a non-root cgroup: pids.max (read-write, the hard limit, default "max"), and the read-only pids.current ("The number of processes currently in the cgroup and its descendants") and pids.events, whose max entry counts how many times the cgroup's total number of processes hit the pids.max limit. Read them in the service's cgroup directory: the CGroup: line of systemctl status names its path, and the files sit under /sys/fs/cgroup/ at that path (for example cat /sys/fs/cgroup/system.slice/app.service/pids.events, where the path is the one the CGroup: line shows). A pids.events max count above zero proves the limit was hit.
5. The system-wide limits.
cat /proc/sys/kernel/threads-max /proc/sys/kernel/pid_max
The proc_sys_kernel(5) page describes threads-max as the "system-wide limit on the number of threads (tasks) that can be created" and pid_max as the value at which PIDs wrap around, which "also acts as a system-wide limit on the total number of processes and threads". Its default is 32768, and on 64-bit systems it can be set up to 2^22.
Fix
Raise a limit only when the count is legitimate. If thread counts are rising without bound, find the process in step 1 and fix that first.
A service limit (TasksMax=)
Save the current definition, including any drop-ins, under a dated name:
systemctl cat app.service > app-service-before-20261120.txt
The systemctl page says cat shows the fragment and drop-in files of units, each preceded by a comment that includes the file name.
Disruptive step: creating a drop-in with a higher TasksMax limit lets a faulty service use more of the host's shared process IDs before it stops, so the fault can spread to other services.
Create the drop-in with the limit. The page says systemctl edit creates a drop-in file and, after editing, configuration is reloaded; --drop-in= names the file (the page's own example is --drop-in=limits.conf, so the name here ends in .conf).
systemctl edit --drop-in=tasksmax-20261120.conf app.service
In the editor, add:
[Service]
TasksMax=8192
The value 8192 is an example, not a recommendation. Check the result with systemctl show --property=TasksMax app.service. A running service may need a restart for a new limit to apply; the systemd pages do not say, so check the shown value after the change, and restart only in a window where its users can tolerate it. To back out, run systemctl cat app.service, which prints the path of the drop-in you created, delete only that file, and run systemctl daemon-reload.
The systemd.exec page says LimitNPROC= limits "the number of processes from one (real) UID" cumulatively and is not enforced for a service running as root, and that "TasksMax= ... is typically a better choice than LimitNPROC=".
A per-user limit from login sessions
The limits.conf(5) page says the pam_limits module applies these limits to user login sessions, with lines of the form domain, type, item, value, and nproc is the "maximum number of processes". An example from the page is @faculty hard nproc 50.
Risk: a higher nproc for an account removes a guard against runaway processes. Save the file first: cp /etc/security/limits.conf /root/limits-conf-saved-20261120, edit it, and note that the change applies to new login sessions. To back out, run cp /root/limits-conf-saved-20261120 /etc/security/limits.conf and start a new session.
System-wide limits
Raise threads-max or pid_max only after a per-user or per-service limit has been ruled out. The proc_sys_kernel(5) page says a value written to threads-max is checked against available RAM. Record the current values first (step 5) and change them only after a per-service limit has been ruled out.
Example (illustrative): On web01, the application's start script (a bash script run by app.service) prints fork: retry: Resource temporarily unavailable in the service's log during a traffic peak, and the application stops creating new workers. Administrators who log in to web01 at the same time have no trouble, because their shells are not inside the service's cgroup. systemctl show --property=TasksMax --property=EffectiveTasksMax app.service reports 4915 for the application, and systemctl status shows its task count at the limit. The service's pids.events file shows a max count above zero. The application had been started with many threads per request. The team sets TasksMax=8192 in a drop-in, and also fixes the application's thread pool size.
Confirm the fix
systemctl show --property=TasksMax app.serviceshows the new value.- The service's task count in
systemctl statusstays below the limit at the next peak. - The
maxentry inpids.eventsstops increasing. - New shells no longer print
fork: retry.
What to watch afterwards
- Alert when a service's task count passes a set share of its limit, not only when it fails.
- Watch the
maxcount inpids.eventsfor services you raised. - Keep
systemctl catoutput of important services in your records. - Treat a rising thread count as a fault to investigate, not a number to keep raising.
Read next
Related articles you may want next.
- Troubleshooting"Too many open files" on a Linux service: raise LimitNOFILE the right wayToo many open files on a Linux service? Read /proc/PID/limits, count descriptors, set LimitNOFILE in a systemd drop-in, and tell a leak from real demand.Read article
- TroubleshootingLinux OOM Killer on a VM: Find the Cause, Set Memory LimitsA Linux VM killed a service for lack of memory? Read the kernel or systemd-oomd record, find which limit applied, then set limits from measured use.Read article
- Troubleshootingsystemd: "Job for nginx.service failed because the control process exited with error code" - find the real errorJob for nginx.service failed because the control process exited? Read systemctl status and journalctl, decode the exit status, test the config.Read article
Risks and limits
- These commands have not been run or lab-tested.
- The retry behaviour is taken from bash source; the bash manual does not describe it.
- The systemd pages do not state whether a running service picks up a new
TasksMax=without a restart. - Container runtimes set their own pids limits and are not covered.
Sources
- fork(2) — Linux man-pages 6.19, 2026-06-05, accessed 20 November 2026
- getrlimit(2) — Linux man-pages 6.19, 2026-02-08, accessed 20 November 2026
- errno(3) — Linux man-pages 6.19, 2026-05-28, accessed 20 November 2026
- proc_sys_kernel(5) — Linux man-pages 6.19, 2026-02-08, accessed 20 November 2026
- Control Group v2 — Linux kernel documentation, page shows "October, 2015", accessed 20 November 2026
- systemd.resource-control(5) — systemd 262~devel (development manual), colophon: obtained 2026-08-04, most recent upstream commit 2026-08-03, accessed 20 November 2026
- systemd.exec(5) — systemd 262~devel (development manual), colophon: obtained 2026-08-04, most recent upstream commit 2026-08-03, accessed 20 November 2026
- systemd-system.conf(5) — systemd 262~devel (development manual), colophon: obtained 2026-08-04, most recent upstream commit 2026-08-03, accessed 20 November 2026
- systemctl — systemd 262 documentation, no date shown, accessed 20 November 2026
- limits.conf(5) — Linux-PAM manual page, 08/04/2026, accessed 20 November 2026
- prlimit(1) — util-linux 2.43, 2026-05-24, accessed 20 November 2026
- ps(1) — procps-ng, 2025-04-23, accessed 20 November 2026
- bash jobs.c — unofficial GNU bash source mirror, file header shows copyright 1989-2015 and no release date, accessed 20 November 2026
This article was originally published on .