Apache log rotation not configured in ISPConfig 3?

Discussion in 'Installation/Configuration' started by gkovacs, Jun 28, 2009.

  1. gkovacs

    gkovacs New Member

    I've just realized that my webserver logs were piling up in the tens of gigabytes. I have checked the logrotate.d config for apache, but it is not configured for the log files in /var/log/ispconfig/httpd/, it only handles the default apache2 log directory, which is empty:

    Code:
    root@ispconfig:/etc/logrotate.d# cat apache2
    /var/log/apache2/*.log {
            weekly
            missingok
            rotate 52
            compress
            delaycompress
            notifempty
            create 640 root adm
            sharedscripts
            postrotate
                    if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
                            /etc/init.d/apache2 reload > /dev/null
                    fi
            endscript
    }
    Furthermore, there is no trace of logrotate in the cron folders or crontab.

    Is ISPConfig able to handle log rotation, or should I configure this myself, separately?

    Logrotate.d seems to pose a problem, as I would have to add every site to it manually, as ISPConfig keeps the log files in separate folders.
     
    Last edited: Jun 28, 2009
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    ISPConfig handles the logfile rotation, so do not use logrotate as it will conflict with ISPConfig.

    The logfiles are cromresssed after one day and removed after 30 days. See the script /usr/local/ispconfig/server/cron_daily.php for details.
     
  3. gkovacs

    gkovacs New Member

    Till, how is it possible that I had just over 5 months of logs, none of it deleted or compressed? We are running ISPConfig 3.0.1 currently.

    The script you mentioned is not in crontab nor is it visible in /etc/cron.daily. Should I copy/link it over?
     
    Last edited: Jun 28, 2009
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    There was a bug ion older release regarding log rotation. You should update your system regularily. The cuurent version is 3.0.1.3.
     
  5. gkovacs

    gkovacs New Member

    Second try

    I have updated my ISPConfig to the latest version, but there is still no log rotation script (or anything connected to ISPConfig whatsoever) in the cron.daily folder or in /etc/crontab.

    Here is the install script output:

    Code:
    Operating System: Debian Lenny/Sid or compatible
    
    This application will update ISPConfig 3 on your server.
    MySQL root password []: xxxxxxxxxxxxxxxx
    
    Reconfigure Services? (yes,no) [yes]: no
    
    Updating ISPConfig
    ISPConfig Port [8080]:
    
    chown: invalid user: `getmail'
    Reconfigure Crontab? (yes,no) [yes]: yes
    
    Updating Crontab
    Update finished.
    Do I need to reconfigure all services to enable the log rotation?
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    Please red my post above. I did not talk about the cron daily folder and I posted you the exact location of the script which does the log rotation and all other nightly tasks.
     
  7. gkovacs

    gkovacs New Member

    Ok, but how does it run if not from cron? How can I set the time it runs, so it would not interfere with my other nightly tasks on the server?

    By the way nothing happened last night. Log files are still there, none is compressed or deleted.
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    As root user run:

    crontab -l

    to get a list of the ispconfig cronjobs.

    The script compresses just the file which is one day old and deletes the file which is one month old. You will have to delete other files manually that are left on your server.
     
  9. gkovacs

    gkovacs New Member

    still no luck

    Well, I let it run for 2 days but nothing happened. The cron job is there, the script runs according to syslog, but it just doesn't compress the day old file and doesn't delete the month old one. Checked at several different sites, with log folders both empty and full with hundreds of files.

    /var/log/ispconfig/cron.log shows nothing relevant.

    What may be wrong? Any ideas how to nail the problem down?
     
  10. Alex1

    Alex1 New Member

    why error.log don't rotate like access.log?
     
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    As far as I know there is no rotation of the error log yet. You can post it as feature request in the bugtracker.
     
  12. gkovacs

    gkovacs New Member

    Till, any idea why on my machine the log rotation would not work?
    Maybe a thing or two to check?
     
  13. till

    till Super Moderator Staff Member ISPConfig Developer

    unpack and copy the attached file to /usr/local/ispconfig/server/ and check if this one works for you. Its the file from the current svn.
     

    Attached Files:

  14. gkovacs

    gkovacs New Member

    I have installed the file you provided, but still nothing: logs don't get compressed or deleted. Any idea what else to check?
     
  15. Keeper

    Keeper New Member

    Looks like I'm bumping an old topic, but forgive me. :)

    I was curious about log rotating too after I saw error logs aren't rotated, but looks like future versions will have that too.

    But I am concerned that cron jobs are in fact php files and thus depend on lenient PHP security settings. On my server for example, I have disabled exec function among others which is used in cron_daily.php.

    Now of course following functions do not work.
    Isn't this a rather severe oversight in ISPConfig 3?
     
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    Not at all. We are talking here about shell scripts and not web scripts. Nothing in ispconfig backend will work if you disable exec for shell scripts of course. You can disable exec in the php.ini for mod_php but not for cli. By the way, disabling exec for cli does not make any sense as all as cli means that the script is executed by someone on the shell anyway.
     
  17. Undo

    Undo New Member

    Hello Till,

    I watched into the given file but it seems that it is not exactly it that handles the log rotation.

    In this file we have the compression of the file once it's already been renamed by another something and the generation of the statistics etc. etc. ...

    So, if you could tell me what is the script / program which is, every day, re-generating the symlink from access.log -> yesterday.log to access.log -> today.log, it would be really neat, because I have a misfunctionning of the log rotation for one website and I would like to see what's hapening and why it is not happening for a particular website.


    Thanks in advance.
     
  18. Undo

    Undo New Member

    Hello guys,

    I've been thinking about all this and finally came with this conclusion : Get rid of Vlogger for Nginx.

    Well, after all, nginx is significantly faster than Apache and it is known to handle correctly disk access.

    But remains the problem of the filenames for Webalizer and AWStats, so I made that :

    Create the directory /var/log/ispconfig/nginx and make nginx log into domaine.tld/access.log as Apache do for the httpd/ folder :

    (configuration Nginx for a given server domain.tld) :

    Code:
    access_log  /var/log/ispconfig/nginx/domain.tld/access.log;
    
    In /etc/logrotate.d/nginx, add at the end of file :

    Code:
    /var/log/ispconfig/nginx/*/access.log {
            daily
            missingok
            nocompress
            rotate 1
            notifempty
            sharedscripts
            postrotate
                    [ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
                    /usr/bin/nohup /usr/local/ispconfig/server/logrotate_nginx.sh >> /var/log/ispconfig/cron.log
            endscript
    }
    Now create the two following files :
    /usr/local/ispconfig/server/logrotate_nginx.sh
    Code:
    #!/bin/sh
    
    PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin
    
    /usr/bin/php -q /usr/local/ispconfig/server/logrotate_nginx.php
    
    /usr/local/ispconfig/server/logrotate_nginx.php
    Code:
    <?php
    
    require('lib/config.inc.php');
    require('lib/app.inc.php');
    
    set_time_limit(0);
    ini_set('error_reporting',E_ALL & ~E_NOTICE);
    
    $yesterday=date('Ymd', time()-86399);
    
    chdir($conf['ispconfig_log_dir']);
    
    $dirs = glob('nginx/*/');
    foreach($dirs as $dir) {
            // Put the content of the access.log of today in the file supposed to be used for yesterday :
            file_put_contents($dir.$yesterday.'-access.log', file_get_contents($dir.'access.log.1'));
            // Empty the current access.log to have only statistics about today at next daily logrotation :
            unlink($dir.'access.log.1');
    }
    
    ?>
    
    Now remains two choices :

    Either you modify server/cron_daily.php in order to go through nginx directories too.
    Either you modify my script and replace "$dir.$yesterday.'.log'" by something like "httpd/".str_replace('nginx/', '', $dir).$yesterday.'-access.log' and this will copy the file into the httpd directory which is parsed by the cron_daily.php
     
    Last edited: Feb 20, 2011

Share This Page