Configuring Your LEMP System (Linux, nginx, MySQL, PHP-FPM) For Maximum Performance

Want to support HowtoForge? Become a subscriber!
 
Submitted by falko (Contact Author) (Forums) on Thu, 2012-09-20 18:03. :: Web Server | MySQL | nginx | PHP

Configuring Your LEMP System (Linux, nginx, MySQL, PHP-FPM) For Maximum Performance

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Follow me on Twitter
Last edited 09/19/2012

If you are using nginx as your webserver, you are looking for a performance boost and better speed. nginx is fast by default, but you can optimize its performance and the performance of all parts (like PHP and MySQL) that work together with nginx. Here is a small, incomprehensive list of tips and tricks to configure your LEMP system (Linux, nginx, MySQL, PHP-FPM) for maximum performance. These tricks work for me, but your mileage may vary. Do not implement them all at once, but one by one and check what effect the modification has on your system's performance.

I do not issue any guarantee that this will work for you!

 

1 Reducing Disk I/O By Mounting PArtitions With noatime And nodiratime

Add noatime and nodiratime to your mount options in /etc/fstab:

vi /etc/fstab

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
# / was on /dev/sda2 during installation
UUID=9cc886cd-98f3-435a-9830-46b316e2a20e /               ext3    errors=remount-ro,noatime,nodiratime,usrjquota=quota.user,grpjquota=quota.group,jqfmt=vfsv0 0       1
# swap was on /dev/sda1 during installation
UUID=bba13162-121d-40a4-90a7-10f78a0097ae none            swap    sw              0       0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto     0       0

#Parallels Shared Folder mount
none         /media/psf   prl_fs   sync,nosuid,nodev,noatime,share,nofail     0       0

Remount the modified partitions as follows (make sure you use the correct mount point for each partition):

mount -o remount /

You can read more about this in this howto: Reducing Disk IO By Mounting Partitions With noatime

 

2 Tuning nginx

2.1 worker_processes

Make sure you use the correct amount of worker_processes in your /etc/nginx/nginx.conf. This should be equal to the amount of CPU cores in the output of

cat /proc/cpuinfo | grep processor

root@server1:~# cat /proc/cpuinfo | grep processor
processor : 0
processor : 1
processor : 2
processor : 3
processor : 4
processor : 5
processor : 6
processor : 7
root@server1:~#

In this example, we have eight CPU cores, so we set

vi /etc/nginx/nginx.conf

[...]
worker_processes 8;
[...]

 

2.2 keepalive_timeout, sendfile, tcp_nopush, tcp_nodelay

Set keepalive_timeout to a sensible value like two seconds. Enable sendfile, tcp_nopush, and tcp_nodelay:

vi /etc/nginx/nginx.conf

[...]
http {
[...]
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 2;
        types_hash_max_size 2048;
        server_tokens off;
[...]
}
[...]

 

2.3 File Cache

Enable the nginx file cache:

vi /etc/nginx/nginx.conf

[...]
http {
[...]
        ##
        # File Cache Settings
        ##

        open_file_cache          max=5000  inactive=20s;
        open_file_cache_valid    30s;
        open_file_cache_min_uses 2;
        open_file_cache_errors   on;
[...]
}
[...]

 

2.4 Enable Gzip Compression

You can read more about Gzip compression here: How To Save Traffic With nginx's HttpGzipModule (Debian Squeeze)

vi /etc/nginx/nginx.conf

[...]
http {
[...]
        ##
        # Gzip Settings
        ##

        gzip on;
        gzip_static on;
        gzip_disable "msie6";
        gzip_http_version 1.1;
        gzip_vary on;
        gzip_comp_level 6;
        gzip_proxied any;
        gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x-js;
        gzip_buffers 16 8k;
[...]
}
[...]

 

2.5 Enable The SSL Session Cache

If you serve https web sites, you should enable the SSL session cache:

vi /etc/nginx/nginx.conf

[...]
http {
[...]
        ssl_session_cache    shared:SSL:10m;
        ssl_session_timeout  10m;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers on;
[...]
}
[...]

 

2.6 Use The FastCGI Cache

If you have cacheable PHP content, you can use the nginx FastCGI cache to cache that content. In your nginx.conf, add a line similar to this one:

vi /etc/nginx/nginx.conf

[...]
http {
[...]
        fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=microcache:10m max_size=1000m inactive=60m;
[...]
}
[...]

The cache directory /var/cache/nginx must exist and be writable for nginx:

mkdir /var/cache/nginx
chown www-data:www-data /var/cache/nginx

(By using tmpfs, you can even place the directory directly in your server's memory which provides another small speed advantage - take a look at this tutorial to learn more: Storing Files/Directories In Memory With tmpfs).

In your vhost configuration, add the following block to your location ~ \.php$ {} section (you can modify it depending on when content should be cached and when not):

[...]
                # Setup var defaults
                set $no_cache "";
                # If non GET/HEAD, don't cache & mark user as uncacheable for 1 second via cookie
                if ($request_method !~ ^(GET|HEAD)$) {
                    set $no_cache "1";
                }
                # Drop no cache cookie if need be
                # (for some reason, add_header fails if included in prior if-block)
                if ($no_cache = "1") {
                    add_header Set-Cookie "_mcnc=1; Max-Age=2; Path=/";
                    add_header X-Microcachable "0";
                }
                # Bypass cache if no-cache cookie is set
                if ($http_cookie ~* "_mcnc") {
                            set $no_cache "1";
                }
                # Bypass cache if flag is set
                fastcgi_no_cache $no_cache;
                fastcgi_cache_bypass $no_cache;
                fastcgi_cache microcache;
                fastcgi_cache_key $scheme$host$request_uri$request_method;
                fastcgi_cache_valid 200 301 302 10m;
                fastcgi_cache_use_stale updating error timeout invalid_header http_500;
                fastcgi_pass_header Set-Cookie;
                fastcgi_pass_header Cookie;
                fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
[...]

So the full location ~ \.php$ {} section could look as follows:

[...]
location ~ \.php$ {

                # Setup var defaults
                set $no_cache "";
                # If non GET/HEAD, don't cache & mark user as uncacheable for 1 second via cookie
                if ($request_method !~ ^(GET|HEAD)$) {
                    set $no_cache "1";
                }
                # Drop no cache cookie if need be
                # (for some reason, add_header fails if included in prior if-block)
                if ($no_cache = "1") {
                    add_header Set-Cookie "_mcnc=1; Max-Age=2; Path=/";
                    add_header X-Microcachable "0";
                }
                # Bypass cache if no-cache cookie is set
                if ($http_cookie ~* "_mcnc") {
                            set $no_cache "1";
                }
                # Bypass cache if flag is set
                fastcgi_no_cache $no_cache;
                fastcgi_cache_bypass $no_cache;
                fastcgi_cache microcache;
                fastcgi_cache_key $scheme$host$request_uri$request_method;
                fastcgi_cache_valid 200 301 302 10m;
                fastcgi_cache_use_stale updating error timeout invalid_header http_500;
                fastcgi_pass_header Set-Cookie;
                fastcgi_pass_header Cookie;
                fastcgi_ignore_headers Cache-Control Expires Set-Cookie;

                try_files $uri =404;
                include /etc/nginx/fastcgi_params;
                fastcgi_pass unix:/var/lib/php5-fpm/web1.sock;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_param PATH_INFO $fastcgi_script_name;
                fastcgi_intercept_errors on;
}
[...]

This would cache pages with the return codes 200, 301, and 302 for ten minutes.

You can read more about this topic here: Why You Should Always Use Nginx With Microcaching

 

2.7 Use FastCGI Buffers

In your vhost configuration, you can add the following lines to your location ~ \.php$ {} section:

[...]
                fastcgi_buffer_size 128k;
fastcgi_buffers 256 16k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
fastcgi_read_timeout 240; [...]

The full location ~ \.php$ {} section could look as follows:

[...]
location ~ \.php$ {
                try_files $uri =404;
                include /etc/nginx/fastcgi_params;
                fastcgi_pass unix:/var/lib/php5-fpm/web1.sock;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_param PATH_INFO $fastcgi_script_name;
                fastcgi_intercept_errors on;

                fastcgi_buffer_size 128k;
                fastcgi_buffers 256 16k;
                fastcgi_busy_buffers_size 256k;
                fastcgi_temp_file_write_size 256k;
                fastcgi_read_timeout 240;
}
[...]

 

2.8 Use memcached

nginx can read full pages directly from memcached. So if your web application is capable of storing full pages in memcached, nginx can fetch that page from memcached. An example configuration (in your vhost) would be as follows:

[...]
        location ~ \.php$ {
                set $no_cache "";
                if ($query_string ~ ".+") {
                        set $no_cache "1";
                }
                if ($request_method !~ ^(GET|HEAD)$ ) {
                        set $no_cache "1";
                }
                if ($request_uri ~ "nocache") {
                        set $no_cache "1";
                }
                if ($no_cache = "1") {
                        return 405;
                }

                set $memcached_key $host$request_uri;
                memcached_pass     127.0.0.1:11211;
                default_type text/html;
                error_page 404 405 502 = @php;
                expires epoch;
        }

        location @php {
                        try_files $uri =404;
                        include /etc/nginx/fastcgi_params;
                        fastcgi_pass unix:/var/lib/php5-fpm/web1.sock;
                        fastcgi_index index.php;
                        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                        fastcgi_param PATH_INFO $fastcgi_script_name;
                        fastcgi_intercept_errors on;
        }
[...]

It is important that your web application uses the same key for storing pages in memcached that nginx uses to fetch these pages from memcached (in this example it's $host$request_uri), otherwise this will not work.

If you store lots of data in memcached, make sure you have allocated enough RAM to memcached, e.g.:

vi /etc/memcached.conf

[...]
# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m 512
[...]

 

2.9 Make Browsers Cache Static Files With The expires Directive

Files (like images, CSS, JS, etc.) that don't change often can be cached by the visitor's browser by using the expires directive (see http://wiki.nginx.org/HttpHeadersModule#expires):

[...]
               location ~*  \.(jpg|jpeg|png|gif|ico)$ {
                         expires 365d;
               }
[...]

 

2.10 Disable Logging For Static Files

Normally it doesn't make much sense to log images or CSS files in the access log. To reduce disk I/O, we can disable logging for such files, e.g. as follows:

[...]
               location ~*  \.(jpg|jpeg|png|gif|ico)$ {
                         log_not_found off;
                         access_log off;
               }
[...]

 

3 Tuning PHP-FPM

3.1 Use A PHP Opcode Cache Like Xcache Or APC

Make sure you have a PHP opcode cache such as Xcache or APC installed. On Debian/Ubuntu, Xcache can be installed as follows:

apt-get install php5-xcache

APC can be installed as follows:

apt-get install php-apc

Make sure you have just one installed (either Xcache or APC), not both. Reload PHP-FPM after the installation:

/etc/init.d/php5-fpm reload

 

3.2 Allocate Enough Memory To Xcache/APC

If you have lots of PHP scripts, you should probably raise the memory that is allocated to Xcache or APC. For Xcache, you can do this in /etc/php5/conf.d/xcache.ini:

vi /etc/php5/conf.d/xcache.ini

[...]
xcache.size  =                512M
[...]

Likewise for APC:

vi /etc/php5/conf.d/apc.ini

[...]
apc.shm_size="512"
[...]

Reload PHP-FPM after your modification:

/etc/init.d/php5-fpm reload

 

3.3 PHP-FPM Emergency Settings

This is more of a reliability setting than a performance setting: PHP-FPM can restart itself if it stops working:

vi /etc/php5/fpm/php-fpm.conf

[...]
; If this number of child processes exit with SIGSEGV or SIGBUS within the time
; interval set by emergency_restart_interval then FPM will restart. A value
; of '0' means 'Off'.
; Default Value: 0
emergency_restart_threshold = 10

; Interval of time used by emergency_restart_interval to determine when
; a graceful restart will be initiated.  This can be useful to work around
; accidental corruptions in an accelerator's shared memory.
; Available Units: s(econds), m(inutes), h(ours), or d(ays)
; Default Unit: seconds
; Default Value: 0
emergency_restart_interval = 1m

; Time limit for child processes to wait for a reaction on signals from master.
; Available units: s(econds), m(inutes), h(ours), or d(ays)
; Default Unit: seconds
; Default Value: 0
process_control_timeout = 10s
[...]

 

3.4 For PHP >= 5.3.9: Use The ondemand Process Manager

If you use PHP >= 5.3.9, you can use the ondemand process manager in a PHP-FPM pool instead of static or dynamic, this will save you some RAM:

[...]
pm = ondemand
pm.max_children = 100
pm.process_idle_timeout = 5s [...]

 

3.5 Use Unix Sockets Instead Of TCP Sockets

To reduce networking overhead, you should configure your pools to use Unix sockets instead of TCP:

[...]
;listen = 127.0.0.1:9000
listen = /var/lib/php5-fpm/www.sock
listen.owner = www-data
listen.group = www-data
listen.mode = 0660 [...]

If you change this, you must of course adjust the location ~ \.php$ {} section in your nginx vhost to use the socket (fastcgi_pass unix:/var/lib/php5-fpm/www.sock; instead of fastcgi_pass 127.0.0.1:9000;):

[...]
location ~ \.php$ {
                try_files $uri =404;
                include /etc/nginx/fastcgi_params;
                ##fastcgi_pass 127.0.0.1:9000;
                fastcgi_pass unix:/var/lib/php5-fpm/www.sock;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_param PATH_INFO $fastcgi_script_name;
                fastcgi_intercept_errors on;
}
[...]

 

3.6 Avoid 502 Bad Gateway Errors With Sockets On Busy Sites

If you use Unix sockets with PHP-FPM, you might encounter random 502 Bad Gateway errors with busy websites. To avoid this, we raise the max. number of allowed connections to a socket. Open /etc/sysctl.conf...

vi /etc/sysctl.conf

... and set:

[...]
net.core.somaxconn = 4096
[...]

Run

sysctl -p

afterwards for the change to take effect.

 

4 Tuning MySQL

4.1 Optimize Your my.cnf

You should use scripts such as mysqltuner.pl or tuning-primer.sh (or both) to find out which settings you should adjust in your my.cnf file. One of the most important variables is query_cache_size, and, if you use InnoDB tables, innodb_buffer_pool_size.

This is an example configuration from a test server with 16GB RAM, about 30 databases with 50% MyISAM tables and 50% InnoDB tables - this worked out quite well for database-driven test sites that were stressed with a benchmark tool (ab):

[...]
key_buffer = 256M

max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 100

table_open_cache = 16384
table_definition_cache = 8192

sort_buffer_size = 256K

read_buffer_size = 128K

read_rnd_buffer_size = 256K

myisam_sort_buffer_size = 64M
myisam_use_mmap = 1
thread_concurrency = 10
wait_timeout = 30

myisam-recover = BACKUP,FORCE

query_cache_limit = 10M
query_cache_size = 1024M
query_cache_type = 1

join_buffer_size = 4M

log_slow_queries        = /var/log/mysql/mysql-slow.log
long_query_time = 1

expire_logs_days        = 10
max_binlog_size         = 100M

innodb_buffer_pool_size = 2048M
innodb_log_file_size = 256M
innodb_log_buffer_size = 16M
innodb_flush_log_at_trx_commit = 0
innodb_thread_concurrency = 8
innodb_read_io_threads = 64
innodb_write_io_threads = 64
innodb_io_capacity = 50000
innodb_flush_method = O_DIRECT
innodb_file_per_table
innodb_additional_mem_pool_size = 256M
transaction-isolation = READ-COMMITTED

innodb_support_xa = 0
innodb_commit_concurrency = 8
innodb_old_blocks_time = 1000
[...]

Please note: If you need ACID compliance, you must set innodb_flush_log_at_trx_commit to 1. You can find out more about this on http://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_flush_log_at_trx_commit.

innodb_io_capacity should be set to high values only if you use MySQL on an SSD. If you use it on a normal hard drive, you better leave that line out.

 

4.2 Use An SSD

You can get a big performance boost by using MySQL with a solid state disk (SSD) as this reduces disk I/O a lot. The easiest way to do this is by mounting the /var/lib/mysql directory to an SSD.

 

5 Web Application Caching

Lots of web applications (such as WordPress with the WP Super Cache or W3 Total Cache plugins, Drupal with the Boost module, TYPO3 with the nc_staticfilecache extension) offer the possibility to create a full page cache which is stored on the hard drive and which can be accessed directly by nginx so that it can bypass the whole PHP-MySQL stack. This provides a huge performance boost.

You can find tutorials about this here:

You can speed the static file cache up even more by placing it directly in the server's memory with the tmpfs filesystem:

Storing Files/Directories In Memory With tmpfs

Of course, you can use tmpfs also for the nginx FastCGI cache from chapter 2.6.

 

6 Links

 

About The Author

Falko Timme is the owner of nginx WebhostingTimme Hosting (ultra-fast nginx web hosting). He is the lead maintainer of HowtoForge (since 2005) and one of the core developers of ISPConfig (since 2000). He has also contributed to the O'Reilly book "Linux System Administration".


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by bobural (not registered) on Thu, 2013-08-29 19:45.
Hi,

 Is it too much to ask for an updated how to... for the new nginx 1.4.2 and php-fpm with a MySQL database? I am trying to adapt this tutorial but I am a bit lost :(

thank you very much...

B.

Submitted by josimar camargo (not registered) on Thu, 2013-07-11 21:59.

very nice post,,,

 very very thanks

Submitted by wahid (registered user) on Sat, 2012-10-20 23:43.

Hi Falko,

Firstly thanks for the How-To!

When I used the suggested config above for APC: apc.shm_size="512" ,  after reloading the php5-fpm process got killed and could not be started, as long as this setting is enabled.

And the log says:

[apc-error] apc_mmap: mmap failed: No space left on device

 I though it's about allocating RAM to the APC, so why it's talking here about "device" ? Anyway my VPS has 1.5GB free RAM.

Any idea?

Wahid 

 

 

Submitted by valentin ciopei (not registered) on Fri, 2012-10-19 19:12.
A verry usefull giude. Thank you
Submitted by Maarten (not registered) on Sat, 2012-09-22 08:46.

Thanks Falko,

I find this a very useful post and it made me evaluate my config files for more improvements!