NGINX 504 Gateway Time-out

Discussion in 'Server Operation' started by onastvar, Feb 20, 2013.

  1. onastvar

    onastvar Member

    After last Nginx update couple of days ago, I'm getting the following 504 Gateway Time-out nginx/1.2.7 when I use any of the web forms (oSticket & WordPress)

    Log
    2013/02/20 23:38:13 [error] 14032#0: *15756 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.122,
    server: mydomain.com, request: "POST /contact/ HTTP/1.1", upstream: "fastcgi://127.0.0.1:9024", host: "mydomain.com", referrer: "http://mydomain.com/contact/"


    Did anything change within nginx, anyone else has this issue, any idea how to fix. Thanks!

    Here's my existing nginx directive:
    Code:
    location / {
                    try_files $uri $uri/ /index.php?$args;
               }
          # Add trailing slash to */wp-admin requests.
           rewrite /wp-admin$ $scheme://$host$uri/ permanent;
    
           location ~*  \.(jpg|jpeg|png|gif|css|js|ico)$ {
                    expires max;
                    log_not_found off;
                }
    location ~* \.php$ {
        fastcgi_pass 127.0.0.1:9001;
        fastcgi_index index.php;
        fastcgi_split_path_info ^(.+\.php)(.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_temp_file_write_size 10m;
        fastcgi_busy_buffers_size 512k;
        fastcgi_buffer_size 512k;
        fastcgi_buffers 16 512k;
        fastcgi_connect_timeout 300;
        fastcgi_send_timeout 300;
        fastcgi_read_timeout 300;
        fastcgi_intercept_errors on;
        fastcgi_next_upstream error invalid_header timeout http_500;
    }
    
     
    Last edited: Feb 21, 2013
  2. falko

    falko Super Moderator Howtoforge Staff Moderator HowtoForge Supporter ISPConfig Developer

    Did you check if a PHP-FPM pool is running on port 9024? Maybe you need to (re)start PHP-FPM.
     
  3. onastvar

    onastvar Member

    Hi Falko, Thanks for your reply. PHP-FPM pool is running on port 9024, I restarted it. Same problem, anything else I should check?
     
  4. falko

    falko Super Moderator Howtoforge Staff Moderator HowtoForge Supporter ISPConfig Developer

    What's the output of
    Code:
    netstat -tap
    ?

    Did you check the PHP-FPM log?
     
  5. onastvar

    onastvar Member

    This is netstat -tap

    Code:
    Active Internet connections (servers and established)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State                                                                                     PID/Program name
    tcp        0      0 *:pop3                  *:*                     LISTEN                                                                                    2580/dovecot
    tcp        0      0 *:imap2                 *:*                     LISTEN                                                                                    2580/dovecot
    tcp        0      0 *:sunrpc                *:*                     LISTEN                                                                                    983/portmap
    tcp        0      0 *:www                   *:*                     LISTEN                                                                                    8091/nginx
    tcp        0      0 *:http-alt              *:*                     LISTEN                                                                                    8091/nginx
    tcp        0      0 *:tproxy                *:*                     LISTEN                                                                                    8091/nginx
    tcp        0      0 localhost.localdom:9010 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 localhost.localdom:9011 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 localhost.localdom:9012 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 *:ftp                   *:*                     LISTEN                                                                                    2699/pure-ftpd (SER
    tcp        0      0 sun.mydomain.com:domain *:*                     LISTEN                                                                                    1841/named
    tcp        0      0 localhost.locald:domain *:*                     LISTEN                                                                                    1841/named
    tcp        0      0 localhost.localdom:9013 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 *:44565                 *:*                     LISTEN                                                                                    995/rpc.statd
    tcp        0      0 *:ssh                   *:*                     LISTEN                                                                                    1861/sshd
    tcp        0      0 localhost.localdom:9014 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 localhost.localdom:9015 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 *:smtp                  *:*                     LISTEN                                                                                    2684/master
    tcp        0      0 localhost.localdoma:953 *:*                     LISTEN                                                                                    1841/named
    tcp        0      0 localhost.localdom:9017 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 localhost.localdom:9018 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 localhost.localdom:9020 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 localhost.localdom:9021 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 localhost.localdom:9023 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 localhost.localdom:9024 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 *:imaps                 *:*                     LISTEN                                                                                    2580/dovecot
    tcp        0      0 *:pop3s                 *:*                     LISTEN                                                                                    2580/dovecot
    tcp        0      0 localhost.localdo:10024 *:*                     LISTEN                                                                                    1840/amavisd (maste
    tcp        0      0 localhost.localdo:10025 *:*                     LISTEN                                                                                    2684/master
    tcp        0      0 localhost.localdom:9033 *:*                     LISTEN                                                                                    7821/php-fpm.conf)
    tcp        0      0 *:mysql                 *:*                     LISTEN                                                                                    2227/mysqld
    tcp        0      0 localhost.localdo:imap2 localhost.localdo:51776 TIME_WAIT                                                                                 -
    tcp        0      0 localhost.localdo:47544 localhost.localdo:mysql ESTABLISHED                                                                               2399/amavisd (ch14-
    tcp        9      0 localhost.localdom:9010 localhost.localdo:57650 CLOSE_WAIT                                                                                7994/php-fpm: pool
    tcp        0      0 localhost.localdo:imap2 localhost.localdo:51777 TIME_WAIT                                                                                 -
    tcp        0      0 localhost.localdo:mysql localhost.localdo:47716 ESTABLISHED                                                                               2227/mysqld
    tcp        0      0 localhost.localdo:imap2 localhost.localdo:51774 TIME_WAIT                                                                                 -
    tcp        0      0 sun.mydomain.:tproxy 192.168.1.122:52726     TIME_WAIT                                                                                 -
    tcp        0      0 localhost.localdom:9024 localhost.localdo:50016 TIME_WAIT                                                                                 -
    tcp        0      0 localhost.localdo:mysql localhost.localdo:47544 ESTABLISHED                                                                               2227/mysqld
    tcp        0      0 localhost.localdo:imap2 localhost.localdo:51778 TIME_WAIT                                                                                 -
    tcp        0      0 localhost.localdo:47716 localhost.localdo:mysql ESTABLISHED                                                                               2400/amavisd (ch13-
    tcp        0      0 sun.mydomain.com:www 192.168.1.122:52725     TIME_WAIT                                                                                 -
    tcp        0      0 sun.mydomain.com:ssh 192.168.1.122:57078     ESTABLISHED                                                                               5765/sshd: webmaste
    tcp6       0      0 [::]:pop3               [::]:*                  LISTEN                                                                                    2580/dovecot
    tcp6       0      0 [::]:imap2              [::]:*                  LISTEN                                                                                    2580/dovecot
    tcp6       0      0 [::]:ftp                [::]:*                  LISTEN                                                                                    2699/pure-ftpd (SER
    tcp6       0      0 [::]:domain             [::]:*                  LISTEN                                                                                    1841/named
    tcp6       0      0 [::]:ssh                [::]:*                  LISTEN                                                                                    1861/sshd
    tcp6       0      0 [::]:smtp               [::]:*                  LISTEN                                                                                    2684/master
    tcp6       0      0 ip6-localhost:953       [::]:*                  LISTEN                                                                                    1841/named
    tcp6       0      0 [::]:imaps              [::]:*                  LISTEN                                                                                    2580/dovecot
    tcp6       0      0 [::]:pop3s              [::]:*                  LISTEN     
    This is my php5-fpm.log

    Code:
    [26-Feb-2013 00:43:03] NOTICE: fpm is running, pid 7327
    [26-Feb-2013 00:43:03] NOTICE: ready to handle connections
    [26-Feb-2013 00:51:02] NOTICE: Reloading in progress ...
    [26-Feb-2013 00:51:02] NOTICE: reloading: execvp("/usr/sbin/php5-fpm", {"/usr/sbin/php5-fpm", "--fpm-config", "/etc/php5/fpm/php-fpm.conf"})
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=7, "/var/lib/php5-fpm/ispconfig.sock"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=7, "/var/lib/php5-fpm/ispconfig.sock"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=9, "127.0.0.1:9013"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=9, "127.0.0.1:9013"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=10, "127.0.0.1:9017"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=10, "127.0.0.1:9017"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=11, "127.0.0.1:9018"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=11, "127.0.0.1:9018"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=12, "127.0.0.1:9020"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=12, "127.0.0.1:9020"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=13, "127.0.0.1:9011"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=13, "127.0.0.1:9011"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=27, "/tmp/php5-fpm.sock"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=27, "/tmp/php5-fpm.sock"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=15, "127.0.0.1:9023"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=15, "127.0.0.1:9023"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=16, "127.0.0.1:9021"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=16, "127.0.0.1:9021"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=17, "127.0.0.1:9033"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=17, "127.0.0.1:9033"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=18, "127.0.0.1:9010"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=18, "127.0.0.1:9010"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=19, "127.0.0.1:9012"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=19, "127.0.0.1:9012"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=20, "/var/lib/php5-fpm/apps.sock"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=20, "/var/lib/php5-fpm/apps.sock"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=22, "127.0.0.1:9015"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=22, "127.0.0.1:9015"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=23, "127.0.0.1:9014"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=23, "127.0.0.1:9014"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=24, "127.0.0.1:9024"
    [26-Feb-2013 00:51:02] NOTICE: using inherited socket fd=24, "127.0.0.1:9024"
    [26-Feb-2013 00:51:02] NOTICE: fpm is running, pid 7821
    [26-Feb-2013 00:51:03] NOTICE: ready to handle connections
    
     
    Last edited: Feb 26, 2013
  6. falko

    falko Super Moderator Howtoforge Staff Moderator HowtoForge Supporter ISPConfig Developer

    Looks ok. Is this a physical server or a virtual machine?
     
  7. onastvar

    onastvar Member

    This is a physical server.
     
  8. falko

    falko Super Moderator Howtoforge Staff Moderator HowtoForge Supporter ISPConfig Developer

    Hm... Have you tried to restart PHP-FPM?
     
  9. onastvar

    onastvar Member

    Yes, restarted PHP-FPM multiple times, still same issue.
     
  10. webelo

    webelo New Member

    I just started experience this on a few VM servers.
    A quick fix that worked was doing a server reboot, but can't be rebooting all the time so gotta find a solution soon.

    My setup is varnish -> nginx -> php-fpm

    I'll be looking into it now if I find anything I'll post here.
     
  11. onastvar

    onastvar Member

    Thanks!
    I rebooted, same issue.
     
  12. falko

    falko Super Moderator Howtoforge Staff Moderator HowtoForge Supporter ISPConfig Developer

    Does it work if you use sockets for PHP-FPM?
     
  13. onastvar

    onastvar Member

    Unfortunately, It doesn't work, I tried both with and without sockets for PHP-FPM.
     
  14. webelo

    webelo New Member

    My problem is no persistant after the reboot it stays good for sometime, its not happening everytime. But I'm testing different configs (even though the setup has been running for the past year without a problem), maybe its a new update thing.

    Try this on your nginx config for the php-fpm

    Change From:
    To:
     
  15. onastvar

    onastvar Member

    Webelo Thanks! Unfortunately, that change also didn't help.

    Since NGINX upgrade to 1.2.7 web forms aren't working and I've been looking to fix this for last two weeks, I don't know if this is NGINX bug or something else. I posted same questions on their forum asked on support chat - still no help!

    I don't know if it's possible to downgrade NGINX to previous version and how, would that even fix the issue?

    I will most likely have to re-install my server with Apache because it's widely used where if there's an issue I'm sure there's a solution available faster.
     
  16. falko

    falko Super Moderator Howtoforge Staff Moderator HowtoForge Supporter ISPConfig Developer

    There seems to be a serious problem with your setup, and I doubt it has anything to do with nginx. It seems more like an issue with PHP-FPM. Have you tried to build and use another PHP version?
     
  17. conductive

    conductive New Member

    Yep 502 Gateway Error.

    I think I went wrong adding the additional php versions via tutorial mid installation.

    Not sure if performing the tutorial again will fix the socket and problem or not.

    2013/07/11 00:36:35 [crit] 26485#0: *27 connect() to unix:/var/lib/php5-fpm/ispconfig.sock failed (2: No such file or directory) while connecting to upstream, client: xx.xx.xx.xx, server: _, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/lib/php5-fpm/ispconfig.sock:", host: "xx.xx.xx.xx:8080"
     

Share This Page