2

I am attempting to spin up 1000x Docker containers via:

nohup docker run --rm --name n-$1-$2 n-v2 bash -c "/root/sw/run.sh $1 $2" 2> ~/logs/log-$1-$2.log &

But after about 500x containers I start getting the following error:

fork: retry: No child processes

The server I am running this on has 48x cores and 192 GB of ram ... I should be able to spin up several thousand of these containers with no problem ... is there a way to increase this limit?

After doing some googling I found that I can check the number of max forks via ulimit -u ... is there a way to change this?

Soren A's answer seems legit, however it doesnt seem like this solved the issue. I can get to ~12287 threads (checed via cat /sys/fs/cgroup/pids/user.slice/user-1000.slice/pids.current) before I start hitting the same issue ... both my hard and soft limits in /etc/security/limits.conf are set to 99999999 yet I am still being capped at around 12287

David Foerster
  • 36,890
  • 56
  • 97
  • 151
CaffeineAddiction
  • 175

1 Answers1

2

You can set the values temporary with ulimit -u <value> Or permanently in /etc/security/limits.conf with lines like

*    hard    nproc  64000
*    soft    nproc  64000

Or the same in a ,conf file in /etc/security/limits.d, ex. /etc/security/limits.d/90-nproc.conf if you like.

Instead of setting it for all users (*) you could do it only for the user starting your processes, so that other users can't grab that many resources on your system.

Soren A
  • 7,191