install monit

Discussion in 'ISPConfig 3 Priority Support' started by bobpit, Jan 19, 2014.

  1. bobpit

    bobpit Member

    I have succesfully installed munin and I am now procceeding to install monit. For monit, I am following this guide: http://www.howtoforge.com/server_monitoring_monit_munin_p2

    1) If monit will monitor the services, restart them if needed and message the admin if something is wrong, right? But I thought that ispconfig also does this:
    Maybe I have to dissable this function in ispconfig?

    2) These messages are sent to some email? Via postfix? Because I am trying to strip the server down. I do not use any email functionality, so I have removed amavis, clamav (spam, antivirus etc).

    3) I want to forward all email warning to my personal email account, eg [email protected]. How do I do this?
     
    Last edited: Jan 19, 2014
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    1) ISPConfig can do this as well, thats right. Monit is just a more advanced option as monit can restart servuces e.g. even if the load gets higher then X. If you use monit, then do not enable the ispconfig service rescue system.

    2) You always need a minimal email system on a server to send messages. You can leave out amavis, clamav, dovecot, spamassassin but not a minimal postfix install as many services require that.

    3) the email address is configured in monit config file, see line:

    set alert [email protected]
     
  3. bobpit

    bobpit Member

    So I UNCHECK "Enable service monitoring and restart on failure" and I CHECK the other 3 options, right?
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Yes, that should be disabled to ensure that not monit and ispconfig try to restart the service which might cause unpredicted results.
     
  5. bobpit

    bobpit Member

    When I start MONIT, I am getting the error:

    Code:
    /etc/monit/monitrc:59: Error: syntax error 'localhost'
    Here are lines 58 and 59 from /etc/monit/monitrc:

    Code:
    set mailserver smtp.gmail.com port 587 username "[email protected]" password "xxxxxx" using tlsv1 with timeout 30 seconds,   # primary mailserver
                   localhost                    # fallback relay
    Here I try to first use GMAIL to send the mail. If this falis, then it will use postfix from localhost, correct? What is wrong with the syntax (I found this in a guide).

    And later on I have this code to send messages to both these addresses:
    Code:
    set alert [email protected]
    set alert [email protected]
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    According to several monit guides, it should work to add another mailserver separated by ",", but your monit instance does not seem to like it. as its very unlikely that gmail is down, you should consider to use just gmail.

    Code:
    set mailserver smtp.gmail.com port 587 username "[email protected]" password "password" using tlsv1 with timeout 30 seconds
     
  7. bobpit

    bobpit Member

    I checked the official syntax and then tried to remember from the University days: http://mmonit.com/monit/documentation/monit.html

    Code:
     SET MAILSERVER {hostname|ip-address [PORT port]
                    [USERNAME username] [PASSWORD password]
                    [using SSLV2|SSLV3|TLSV1] [CERTMD5 checksum]}+ 
                    [with TIMEOUT X SECONDS]
                    [using HOSTNAME hostname]
    The following did not give me any error messages:

    Code:
    set mailserver smtp.gmail.com port 587 username "[email protected]" password "xxxxxx" using tlsv1,    # primary mailserver
                   localhost                    # fallback relay
                   with timeout 30 seconds
    Obviously, the "with timeout 30 seconds" had to go at the end, after all mail servers.

    Thank you for the brainstorming. It was good that you reasured me I was on the right track and not totally wrong.
     
  8. bobpit

    bobpit Member

    Till

    Since I use nginx 1.4.4 instead of Apache, can you help me rewrite the nginx part of /etc/monit/monitrc ?

    The tutorial (http://www.howtoforge.com/server_monitoring_monit_munin_p2) says:
    Code:
    check process apache with pidfile /var/run/apache2.pid
       group www
       start program = "/etc/init.d/apache2 start"
       stop program  = "/etc/init.d/apache2 stop"
       if failed host www.example.com port 80 protocol http
          and request "/monit/token" then restart
       if cpu is greater than 60% for 2 cycles then alert
       if cpu > 80% for 5 cycles then restart
       if totalmem > 500 MB for 5 cycles then restart
       if children > 250 then restart
       if loadavg(5min) greater than 10 for 8 cycles then stop
       if 3 restarts within 5 cycles then timeout
    I managed to write this:
    Code:
    check process nginx with pidfile /var/run/nginx.pid
    start program = "/etc/init.d/nginx start"
    stop program = "/etc/init.d/nginx stop"
    if failed host 127.0.0.1 port 80 then restart
    if cpu is greater than 40% for 2 cycles then alert
    Fot a test file, I have made https://surf-anonymous.info/test/index.html

    Also, what am I supposed to enter for email address here (/var/certs/monit.cnf):
    Code:
    emailAddress_default            = [email protected]
    After starting MONIT, I started getting to my gmail aggress these emails:
    Code:
    Delivery to the following recipient failed permanently:
    
         [email protected]
    
    Technical details of permanent failure:
    DNS Error: Domain name not found
    Most probably, at some configuration file I entered [email protected], while it should be a more meaningfull [email protected]?

    I am also getting these emails:
    Code:
    Monit instance changed Service myhost.mydomain.tld
    
            Date:        Wed, 22 Jan 2014 23:56:13
            Action:      start
            Host:        server1.surf-anonymous.info
            Description: Monit started
    
    Your faithful employee,
    Monit
    Do I have to replace "myhost.mydomain.tld" with something else at some configuration file too?
     
    Last edited: Jan 23, 2014
  9. till

    till Super Moderator Staff Member ISPConfig Developer

  10. bobpit

    bobpit Member

    ok, I inserted the nginx section. monit is working and it does monitor nginx.

    issue 1: When I reset the VPS, I got the following undelivered emails (along with the correct ones delivered at [email protected]):

    Code:
    Delivery to the following recipient failed permanently:
    
         [email protected]
    
    Technical details of permanent failure:
    DNS Error: Domain name not found
    
    This is the code I use in monitrc:

    Code:
    # send alert emails to both addresses 
    set alert [email protected]
    set alert [email protected]
    I copied "set alert [email protected]" from the guide. What am I doing wrong here?


    issue 2:
    How do I monitor php5-fpm too?

    I copied the following code but it does not work because this is invalid "/tmp/php-fpm.sock":
    Code:
    check process php5-fpm with pidfile /var/run/php5-fpm.pid
       group php 
       start program = "/etc/init.d/php5-fpm start"
       stop program  = "/etc/init.d/php5-fpm stop"
       if failed unixsocket /tmp/php-fpm.sock then restart
       if 4 restarts within 5 cycles then timeout
       depends on nginx
    After searching the configuration files, I found the following files in this directory "/var/lib/php5-fpm/":
    web2.sock
    ispconfig.sock
    apps.sock

    issue 3:
    When things start to fail, I'd like to REBOOT the server. How do I do this?

    I did copy the commands like "if 4 restarts within 5 cycles then timeout", but the manual would not tell me what this "timeout" action does really. What I really want is :
    if 4 restarts within 5 cycles then REBOOT THE SERVER
     
    Last edited: Jan 26, 2014
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    1) Check that localhost is listed in /etc/hosts and that postfix is installed on your server and that it has localhost listed in the mydestination line.

    2) On Debian Wheez, the socket of the default php-fpm pool is: /var/run/php5-fpm.sock

    3) use reboot command as start command.
     
  12. bobpit

    bobpit Member

    issue 1:
    I made several changes, I still have the same problem. How can I see if [email protected] is setup correctly to receive emails?

    contents of my HOSTS file:
    Code:
    127.0.0.1 localhost.localdomain localhost
    # Auto-generated hostname. Please do not remove this comment.
    192.210.214.129 server1.surf-anonymous.info  server1
    ::1 localhost.localdomain localhost
    Code:
    [email protected]:~# sudo postfix status
    postfix/postfix-script: the Postfix mail system is running: PID: 706
    Code:
    [email protected]:~# sudo postconf mydestination
    mydestination = server1.surf-anonymous.info, localhost, localhost.localdomain
    So I decided to reinstall postfix, by following this guide: https://www.digitalocean.com/community/articles/how-to-install-and-setup-postfix-on-ubuntu-12-04

    I created this file: /etc/postfix/virtual
    tha contains the following line:
    [email protected] root

    I tried it and again I have the following message from monit:
    Code:
    Delivery to the following recipient failed permanently:
    
         [email protected]
    
    Technical details of permanent failure:
    DNS Error: Domain name not found
    does this help?
    Code:
    [email protected]:~# grep -r -l "[email protected]" /etc/*
    grep: /etc/blkid.tab: No such file or directory
    /etc/fail2ban/jail.conf
    grep: /etc/mail/smrsh/mail.local: No such file or directory
    /etc/mail/submit.cf
    /etc/monit/monitrc
    /etc/warnquota.conf
    issue 2:
    I have ubuntu 12.04. This file does not exist: /var/run/php5-fpm.sock

    issue 3:
    You mean to replace this
    Code:
    if 3 restarts within 5 cycles then [B]timeout[/B]
    with this?
    Code:
    if 3 restarts within 5 cycles then [B]reboot[/B]
     
    Last edited: Jan 28, 2014
  13. till

    till Super Moderator Staff Member ISPConfig Developer

    1) You will see it in your mail.log. stop a service to simulate a failure, then you should see new lines in your mail.log file for a emails ent to the root user.

    2) No. Normally you use a start command like this:

    "/etc/init.d/abc start"

    so instead of starting / sopping a service, use the reboot command:

    "reboot"
     
  14. bobpit

    bobpit Member

    issue 3:

    How come the "reboot' command is not mentioned in the monit manual? http://mmonit.com/monit/documentation/monit.html

    What I have is this:
    Code:
    check process nginx with pidfile /var/run/nginx.pid
    start program = "/etc/init.d/nginx [B]start[/B]"
    stop program = "/etc/init.d/nginx stop"
    if failed host 127.0.0.1 port 80 then restart
    if cpu is greater than 40% for 2 cycles then alert
    You suggest that I change it to this:
    Code:
    check process nginx with pidfile /var/run/nginx.pid
    start program = "/etc/init.d/nginx [B]reboot[/B]"
    stop program = "/etc/init.d/nginx stop"
    if failed host 127.0.0.1 port 80 then restart
    if cpu is greater than 40% for 2 cycles then alert
    From what I understand, the effect of this is:
    1) if nginx is not working it will reboot the entire server
    2) if nginx is loaded but failing to respond properly (if failed host 127.0.0.1 port 80 then restart) then reboot the server.

    Is this what you mean?

    Well, I do not want #1 to happen. If a process is not loaded, I want it to load (START) without rebooting. If several restarts fail, then something is wrong with the server so I want the server rebooted. This is why I was thinking of something allong the lines of
    Code:
    if 3 restarts within 5 cycles then reboot
    Obviously this code does not work. But if you understand what I am trying to do, what would be the command for this?
     
  15. till

    till Super Moderator Staff Member ISPConfig Developer

    reboot is a linux command, not a monit command. The monit manual does not list all programs that are availble on the shell of a linux server.

    No. I said that you should use the reboot command instead of the init start script, if you want to reboot the server (which is not what I would do on my server ebtw).

    start program = "reboot"
     
  16. bobpit

    bobpit Member

    So you are suggesting this:
    Code:
    check process nginx with pidfile /var/run/nginx.pid
       start program = "[B]reboot[/B]" 
       stop  program = "/etc/init.d/nginx stop"
       if failed host 127.0.0.1 port 80 then restart
    Obviously I do not want this.

    I will first try to start the service. So the following code is good, for what it does:
    Code:
    check process nginx with pidfile /var/run/nginx.pid
       start program = "/etc/init.d/nginx start"
       stop  program = "/etc/init.d/nginx stop"
       if failed host 127.0.0.1 port 80 then restart
    
    What I originally asked you is how can I reboot the server if nginx or any other process repeatedly fails after many restarts. Something like this:
    Code:
    check process nginx with pidfile /var/run/nginx.pid
       start program = "/etc/init.d/nginx start"
       stop  program = "/etc/init.d/nginx stop"
       if failed host 127.0.0.1 port 80 then restart
       if 4 restarts within 5 cycles then [B]reboot[/B]
    Well, I just tried it, it does not work:
    Code:
    [email protected]:~# monit -t
    /etc/monit/monitrc:157: Error: [B]syntax error 'reboot'[/B]
    So how do I do this?
     
    Last edited: Jan 28, 2014
  17. till

    till Super Moderator Staff Member ISPConfig Developer

    Try e.g.:

    if 4 restarts within 5 cycles then exec /sbin/reboot
     
  18. bobpit

    bobpit Member

    issue 3:
    ok, "monit -t" did not complain about it. Now I will have to test it. Thank.

    issue 2:
    I am still stuck with monitoring php5-fpm. I do not know how to test if it is working. It might work if I could find the file php-fpm.sock.
     
  19. till

    till Super Moderator Staff Member ISPConfig Developer

    I posted in #11 of this thread were the .socjk file on Ubuntu and Debian is. If you modífied your config that php-fpm creates the socj file in another location, then take a look into the php-fpm pool.d directory for the config file of the www pool, it contains all paths that php-fpm uses.
     
  20. bobpit

    bobpit Member

    issue 3:

    I went to file /etc/php5/fpm/pool.d/www.conf and I replaced this line
    Code:
    listen = 127.0.0.1:9000
    with this line:
    Code:
    listen = /var/run/php5-fpm.sock
    After restarting php5-fpm the unix socket file was created for php5-fpm here:
    Code:
    /var/run/php5-fpm.sock
    More details here: bbb-solutions.blogspot.com/2014/01/where-is-ubuntu-php5-fpm-unix-socket.html

    After that it was trivial to configure monit:
    Code:
     check process php5-fpm with pidfile /var/run/php5-fpm.pid
       group php 
       start program = "/etc/init.d/php5-fpm start"
       stop program  = "/etc/init.d/php5-fpm stop"
       if failed unixsocket /var/run/php5-fpm.sock then restart
       if 4 restarts within 5 cycles then exec /sbin/reboot
       depends on nginx
    It looks easy after you know what you are doing.

    Thanks Till
     
    Last edited: Jan 28, 2014

Share This Page