mail.log rotation

Discussion in 'Installation/Configuration' started by AlArenal, Jun 29, 2007.

  1. AlArenal

    AlArenal New Member

    I'm sorry for crossposting, but I need a solution to a problem induced by ISPConfig.

    I'm using Debian Etch with ISPC and Postfix on which by default the mail logs are rotated by syslog's cronjob around 06:30 every morning. Problem is, that at 23:00 an ISPC cronjob starts mail_logs.php which does something to mail.log (gather statistics and stuff), copying it from here to there, starting a new one, ... pretty weird and not really documented stuff.

    Can't you use something more spohisticated, leaving the original log files as they are, so ISPC's cron job does not interfer with other cron jobs?

    In the meantime:
    Is mail_logs.php something so cool I wouldn't want to live without it running every night or may I just throw it out of my crontab?

    Thx :)
  2. falko

    falko Super Moderator ISPConfig Developer

    You can remove it from your crontab if you don't care about email traffic statistics in ISPConfig. :)
  3. AlArenal

    AlArenal New Member

    That's cool, thank you!

    BTW, do you plan to redesign this part in ISPConfig 3?
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    I never seen your problem on any other installation, so I guess it must have something to do that cronjobs on your server collide. Please try to run the conjobs at another time so they dont collide.

    ISPconfig 3 has a new virtual mail system.
  5. Morons

    Morons Member

    Traffic Statistics

    What traffic statistics, how do I get that statistics you speak about?:confused:
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    The mail traffic statistics are on the statistics tab of the website in ISPConfig like all other traffic stats :)
  7. Morons

    Morons Member

    That statistics for mail is in any case totally not functioning and soe time show zero value, the reason is that ISP rotate the logfiles at mindnite, causing syslog sometime NOT to work at all till the next day 06:23! when syslog can restart it. I saw these gaps in logfiles as I analyse them with some scripts but found them unreliable and full of gaps! Thus far I could not detect the villan! Gotcha!:D
  8. Anwe

    Anwe New Member

    Would just like to add...

    We are also experiencing this problem. Each day the mail.log gets rotated at 23:59 by ISPConfig and then again by logrotate at 06:25. We're using Debian Etch, Postfix (from etch) and ISPConfig 2.2.13.

    Is there a simple way to modify the script to work on a temporary copy? I would rather not deactivate the traffic statistics, customers would probably complain.

    Also the mail_logs.php script won't compress old logs, this seems a bit wasteful. I noticed my maillogs for this machine are above 5 GB since about may :) I do monitor so i would've caught this before it became a problem, but i tend to assume logs are rotated correctly.

    Kind regards
    Andreas Wettergren
  9. falko

    falko Super Moderator ISPConfig Developer

    I've added this to our bugtracker.
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    I checked the mail logs script. ISPConfig works already on a temporary copy and not on the original mail logfile.

    ISPConfig does not rotate logs, its just analyzes a temporary copy. You should check your log rotate settings to ensure that the mail log is compressed.
  11. lewis_saint

    lewis_saint New Member

    I too am having problems. All of my rotated mail.log files start at 23:59 and end at 06:25. It is as if ISPC deletes all prior content in the log file when it analyses it at 23:59. Till, falko, any suggestions?
  12. xrat

    xrat Member

    Eventually, I took the time to investigate this a bit more. My Debian box with ISPconfig is running Debian 4.0 "Etch" (pretty much a default installation) with a current ISPconfig v2.2.19.

    The problem persists: /var/log/mail.log.0 shows only lines from midnight until approximately 6:30. The reason is, vaguely put, a conflict between default Debian log file rotation and ISPconfig, more specifically its cronjob
    59 23 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/mail_logs.php
    BTW, if you miss your data have a look at the other mail.* log files, especially /var/log/

    Whether your Debian log file rotation kicks in or not you can check with
    syslogd-listfiles --weekly
    It returns all files which get rotated weekly by means of /etc/cron.weekly/sysklogd which again, on my system, is run every Sunday morning at 6:47 (see /etc/crontab).

    But the main problem I see -- sorry to say this -- in ISPconfig's /root/ispconfig/scripts/shell/mail_logs.php -- though the whole issue is definitely not trivial. Anyway, from my point of view, mail_logs.php needs some rethinking.

    As till wrote, mail_logs.php is working on a temporary copy. I guess he referred to line 53
    $mod->log->caselog("cp -f $dist_mail_log $dist_mail_log.$datum", $FILE, __LINE__);
    So this is true. But a few lines further below the original data is overwritten with
    $fp = fopen($dist_mail_log, "w");
    fwrite($fp, "");
    There our data is lost. mail_logs.php deletes everything until midnight.

    I do not know enough about ISPconfig but I am afraid this here is bound to break on systems with log rotation that occurs during the day. I guess it also means that ISPconfig's mail stats are not accurate. Certainly, just disabling the lines which erase $dist_mail_log does not help.

    Nevertheless, maybe there is a simpler fix since till reported that he has never seen this problem, probably for a reason.

    -- xrat

    P.S.: In mail_logs.php there are references to $datum2 which I assume does not exist/is empty. It does not break the script but I recommend to fix it anyway should the code ever be used somewhere else.
  13. xrat

    xrat Member

    A quick fix

    When I wrote this I missed the fact that mail.log is also rotated on a daily basis at 6:25 by means of /etc/crontab -> /etc/cron.daily/sysklogd

    A quick fix to the issues pointed out above is the following:

    Make syslog to log all mail.* data to another log file which is then used only by ISPconfig. E.g. add to /etc/syslog.conf the line
    mail.*    -/var/log/mail.log4ispc
    Create a copy of the already existing log file with the new name and reload syslog's configuration with
    cd /var/log
    cp -a mail.log mail.log4ispc && /etc/init.d/sysklogd reload
    Then change the log file name in ISPconfig. Login as admin and go to Management > Server > Settings. On tab "EMail" change field "Log File" from (I suppose, at least for Debian) /var/log/mail.log to /var/log/mail.log4ispc

    Note that with these settings also /var/log/mail.log4ispc will get automagically rotated by Debian's sysklogd but only weekly. Thus, every Sunday the mail stats of ISPconfig will miss 6 hours.

    Please check yourself if the here suggested changes are suitable for your system.

    -- xrat
  14. Morons

    Morons Member

    Proper Cron!

    Best practice would be to include the cronjobs into the /etc/cron.daily, /etc/cron.weekly etc.
    If we can get all the ispc cron scripts under /etc/cron.daily that process all the daily ispc activity then the system will take care of the timing and rotations will be synchronized.
  15. lewis_saint

    lewis_saint New Member

    Is there a simple fix possible for /root/ispconfig/scripts/shell/mail_logs.php?

  16. wpwood3

    wpwood3 New Member

    I am having this same problem.
    My mail logs do not include a full 24 hour day.

    Is there a fix in the works?
  17. xrat

    xrat Member

    Dude, it's up there!
  18. lewis_saint

    lewis_saint New Member

    By your own words xrat, the fix improves the situation, but does not fully rectify the problem.


  19. wpwood3

    wpwood3 New Member

    Dude...that's a hack. Not a fix.

    A fix means that the bug in ISPConfig is addressed so the problem does not occur.
  20. till

    till Super Moderator Staff Member ISPConfig Developer

    We will have a look at this. Falko already added it to the bugtracker.

Share This Page