More than 6% of the web is powered by nginx today. Nginx out of the box is very fast and extremely good at serving static content. But then you have a growing traffic you need to monitor all of your infrastructure and make sure no body is gating your speed.

Few tips to scale up nginx

The first thing you need to monitor is your error.log. Nginx spews out all the errors into your error.log. Make sure you set the error.log path in your conf file or while configure. The first thing you might notice is to increase the number of nginx workers -> worker_processes 4;. This will spawn 4 workers to handle your traffic. In most cases there is no need to go beyond 2., but if you have a lot of traffic make sure you match this value to the number of cores on your machine.

After your nginx starts to handle a lot of connections you might start to see this in your error.log

2011/05/09 12:32:36 [alert] 4671#0: accept() failed (24: Too many open files)
2011/05/09 16:13:55 [alert] 7245#0: 1024 worker_connections are not enough

By default nginx is allowed to have only 1024 file descriptors at a time. If it exceeds that limit bounce the number of file descriptors your machine can handle. If you are on ubuntu edit your /etc/security/limits.conf file. Add this just before end of the file

*       soft    nofile  16384
*       hard    nofile  32768
# End of file

To apply the changes in /etc/security/limits.conf on Ubuntu, add the following line in the /etc/pam.d/common-session file:

session required  pam_limits.so

By this your are extending the number of file descriptors your machine can handle. After that change edit your nginx.conf to make your nginx handle more file descriptors.

worker_rlimit_nofile 32768;
events {
    worker_connections  4096;
    use epoll;
}

You can also increase your worker_connections to handle more connections at a time. Make sure you read nginx docs before you change more.

If you are using nginx as a proxying server make sure your backends are up or else you might end up with **connect() failed (111: Connection refused) while connecting to upstream, client: xx.xx.xx.xx **.

Once you past the default file descriptor limit it is better to move the static stuff into a CDN or else make sure there is a specfic nginx server serving just the static.

And make sure you monitor nginx with munin. I use munin-nginx plugin

server {
   listen 127.0.0.1;
   server_name localhost;
   location /nginx_status {
           stub_status on;
           access_log   off;
           allow 127.0.0.1;
           deny all;
   }

## In your munin plugin-conf.d/munin
[nginx*]
user root
env.url http://localhost/nginx_status