#1  
Old 3rd March 2011, 22:25
8omas 8omas is offline
Senior Member
 
Join Date: Jan 2011
Posts: 148
Thanks: 16
Thanked 35 Times in 26 Posts
Default Awstats anytime update

Below is a cut off, of the default cron file of ISPConfig 3.
Till, can we use it to update awstats, as frequently as we want?

My replacements:
- I removed every logrotate action you had and everything else, that had to do with modifying files
- I erased every yesterday variable and I used the default access.log for each website
- I tested it.

As I see after the tests, when I run it , then the stats REPLACED and updated for today. When the real cron runs every night, then the stats created with the above script, replaced again bringing us back to the state that ISPConfig wants. If (after 00:30) I run the script again, then the stats replaced again, but actually only the 'new' day is modified.

I think it works. Can I have your opinion?


Code:
<?php
require('/usr/local/ispconfig/server/lib/config.inc.php');
require('/usr/local/ispconfig/server/lib/app.inc.php');

set_time_limit(0);

// make sure server_id is always an int
$conf['server_id'] = intval($conf['server_id']);


// Load required base-classes
$app->uses('ini_parser,file,services,getconf');


#######################################################################################################
// Create awstats statistics
#######################################################################################################
$sql = "SELECT domain_id, domain, document_root FROM web_domain WHERE stats_type = 'awstats' AND server_id = ".$conf['server_id'];
$records = $app->db->queryAllRecords($sql);

$web_config = $app->getconf->get_server_config($conf['server_id'], 'web');

foreach($records as $rec) {
        $logfile = escapeshellcmd($rec['document_root'].'/log/access.log');

        $domain = escapeshellcmd($rec['domain']);
        $statsdir = escapeshellcmd($rec['document_root'].'/web/stats');
        $awstats_pl = $web_config['awstats_pl'];
        $awstats_buildstaticpages_pl = $web_config['awstats_buildstaticpages_pl'];
        
        $awstats_conf_dir = $web_config['awstats_conf_dir'];
        $awstats_website_conf_file = $web_config['awstats_conf_dir'].'/awstats.'.$domain.'.conf';
        
        if(is_file($awstats_website_conf_file)) unlink($awstats_website_conf_file);
        
        if(!is_file($awstats_website_conf_file)) {
                $awstats_conf_file_content = 'Include "'.$awstats_conf_dir.'/awstats.conf"
LogFile="/var/log/ispconfig/httpd/'.$domain.'/access.log"
SiteDomain="'.$domain.'"

HostAliases="www.'.$domain.' localhost 127.0.0.1"';
                file_put_contents($awstats_website_conf_file,$awstats_conf_file_content);
        }

        if(!@is_dir($statsdir)) mkdir($statsdir);

        // awstats_buildstaticpages.pl -update -config=mydomain.com -lang=en -dir=/var/www/domain.com/web/stats -awstatsprog=/path/to/awstats.pl
        $command = "$awstats_buildstaticpages_pl -update -config='$domain' -lang=en -dir='$statsdir' -awstatsprog='$awstats_pl'";

        if($awstats_pl != '' && $awstats_buildstaticpages_pl != '' && fileowner($awstats_pl) == 0 && fileowner($awstats_buildstaticpages_pl) == 0) {
                exec($command);
                rename($rec['document_root'].'/web/stats/awstats.'.$domain.'.html',$rec['document_root'].'/web/stats/index.html');
                $app->log('Created awstats statistics with command: '.$command,LOGLEVEL_DEBUG);
        } else {
                $app->log("No awstats statistics created. Either $awstats_pl or $awstats_buildstaticpages_pl is not owned by root user.",LOGLEVEL_WARN);
        }

}

die("finished.\n");
?>
I named the file as /usr/local/ispconfig/server/mycron.php and I can run it as
Code:
/usr/bin/php -q /usr/local/ispconfig/server/mycron.php

Last edited by 8omas; 3rd March 2011 at 22:32.
Reply With Quote
The Following User Says Thank You to 8omas For This Useful Post:
falko (4th March 2011)
Sponsored Links
  #2  
Old 4th March 2011, 20:09
8omas 8omas is offline
Senior Member
 
Join Date: Jan 2011
Posts: 148
Thanks: 16
Thanked 35 Times in 26 Posts
Default

Ok.
I think I confirmed it. I run it for about 2-3 days now and it works!
I can update awstats anytime I want (e.g. through cron)

By the way: I can have Greek (that means ... any language that awstats support) just by changing the line:
Code:
$command = "$awstats_buildstaticpages_pl -update -config='$domain' -lang=en -dir='$statsdir' -awstatsprog='$awstats_pl'";
with this one
Code:
$command = "$awstats_buildstaticpages_pl -update -config='$domain' -lang=gr -dir='$statsdir' -awstatsprog='$awstats_pl'";
Actually, I have to run it, once, after the default cron (e.g. 00:40). That means, that I can have Greek and frequently updated stats WITHOUT even change a single line of the ISPConfig php files.

A suggestion:
Maybe ISPConfig could have an implementation like this. The default cron.daily could call a script to update stats and after that make the necessary rotations. With this, we could call the script anytime we want. Even through ISPConfig interface via a button.
Reply With Quote
  #3  
Old 6th November 2011, 23:34
8omas 8omas is offline
Senior Member
 
Join Date: Jan 2011
Posts: 148
Thanks: 16
Thanked 35 Times in 26 Posts
Default

The previous code applies only up to ISPConfig 3.0.3. For ISPConfig 3.0.4 I updated the code in the "Extending perfect server" tutorial.
I did it today, so I have not test it enough.
Reply With Quote
  #4  
Old 2nd December 2011, 22:23
8omas 8omas is offline
Senior Member
 
Join Date: Jan 2011
Posts: 148
Thanks: 16
Thanked 35 Times in 26 Posts
Default

I re-updated the tutorial with the following code

Code:
<?php

/*
Copyright (c) 2007, Till Brehm, projektfarm Gmbh
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright notice,
      this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright notice,
      this list of conditions and the following disclaimer in the documentation
      and/or other materials provided with the distribution.
    * Neither the name of ISPConfig nor the names of its contributors
      may be used to endorse or promote products derived from this software without
      specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

require('/usr/local/ispconfig/server/lib/config.inc.php');
require('/usr/local/ispconfig/server/lib/app.inc.php');

set_time_limit(0);

// make sure server_id is always an int
$conf['server_id'] = intval($conf['server_id']);


// Load required base-classes
$app->uses('ini_parser,file,services,getconf');


#######################################################################################################
// Create awstats statistics
#######################################################################################################

$sql = "SELECT domain_id, domain, document_root FROM web_domain WHERE stats_type = 'awstats' AND server_id = ".$conf['server_id'];
$records = $app->db->queryAllRecords($sql);

$web_config = $app->getconf->get_server_config($conf['server_id'], 'web');


foreach($records as $rec) {
        $logfile = escapeshellcmd($rec['document_root'].'/log/access.log');
        $domain = escapeshellcmd($rec['domain']);
        $statsdir = escapeshellcmd($rec['document_root'].'/web/stats');
        $awstats_pl = $web_config['awstats_pl'];
        $awstats_buildstaticpages_pl = $web_config['awstats_buildstaticpages_pl'];

        $awstats_conf_dir = $web_config['awstats_conf_dir'];
        $awstats_website_conf_file = $web_config['awstats_conf_dir'].'/awstats.'.$domain.'.conf';

        if(is_file($awstats_website_conf_file)) unlink($awstats_website_conf_file);

        $sql = "SELECT domain FROM web_domain WHERE (type = 'alias' OR AND type = 'subdomain') server_id = ".$conf['server_id'];
        $aliases = $app->db->queryAllRecords($sql);
        $aliasdomain = '';

        if(is_array($aliases)) {
                foreach ($aliases as $alias) {
                        $aliasdomain.= ' '.$alias['domain']. ' www.'.$alias['domain'];
                }
        }

        if(!is_file($awstats_website_conf_file)) {
                $awstats_conf_file_content = 'Include "'.$awstats_conf_dir.'/awstats.conf"
LogFile="/var/log/ispconfig/httpd/'.$domain.'/access.log"
SiteDomain="'.$domain.'"
HostAliases="www.'.$domain.' localhost 127.0.0.1"'.$aliasdomain;
                file_put_contents($awstats_website_conf_file,$awstats_conf_file_content);
        }

if(!@is_dir($statsdir)) mkdir($statsdir);

if (date("d") == 1) {

        $awmonth = date("m")-1;
        $awyear = date("Y");
        if (date("m") == 1) {
                $awyear = date("Y")-1;
                $awmonth = "12";
        }

        $statsdirold = $statsdir."/".$awyear."-".$awmonth."/";

        if (!is_dir($statsdirold)) {
                mkdir($statsdirold);
                $command = "$awstats_buildstaticpages_pl -month='$awmonth' -year='$awyear' -update -config='$domain' -lang=".$conf['language']." -dir='$statsdir' -awstatsprog='$awstats_pl'";

                if($awstats_pl != '' && $awstats_buildstaticpages_pl != '' && fileowner($awstats_pl) == 0 && fileowner($awstats_buildstaticpages_pl) == 0) {
                        exec($command);
                        if(is_file($rec['document_root'].'/web/stats/index.html')) unlink($rec['document_root'].'/web/stats/index.html');
                        rename($rec['document_root'].'/web/stats/awstats.'.$domain.'.html',$rec['document_root'].'/web/stats/awsindex.html');
                        if(!is_file($rec['document_root']."/web/stats/index.php")) copy("/usr/local/ispconfig/server/conf/awstats_index.php.master",$rec['document_root']."/web/stats/index.php");

                        $app->log('Created awstats statistics with command: '.$command,LOGLEVEL_DEBUG);
                } else {
                        $app->log("No awstats statistics created. Either $awstats_pl or $awstats_buildstaticpages_pl is not owned by root user.",LOGLEVEL_WARN);
                }

                $files = scandir($statsdir);
                foreach ($files as $file) {
                        if (substr($file,0,1) != "." && !is_dir($file) && substr($file,0,1) != "w" && substr($file,0,1) != "i") copy("$statsdir"."/"."$file","$statsdirold"."$file");
                }
        }
}

       $awmonth = date("n");
       $awyear = date("Y");

       $command = "$awstats_buildstaticpages_pl -month='$awmonth' -year='$awyear' -update -config='$domain' -lang=".$conf['language']." -dir='$statsdir' -awstatsprog='$awstats_pl'";

       if($awstats_pl != '' && $awstats_buildstaticpages_pl != '' && fileowner($awstats_pl) == 0 && fileowner($awstats_buildstaticpages_pl) == 0) {
                exec($command);
                if(is_file($rec['document_root'].'/web/stats/index.html')) unlink($rec['document_root'].'/web/stats/index.html');
                rename($rec['document_root'].'/web/stats/awstats.'.$domain.'.html',$rec['document_root'].'/web/stats/awsindex.html');
                if(!is_file($rec['document_root']."/web/stats/index.php")) copy("/usr/local/ispconfig/server/conf/awstats_index.php.master",$rec['document_root']."/web/stats/index.php");

                $app->log('Created awstats statistics with command: '.$command,LOGLEVEL_DEBUG);
        } else {
                $app->log("No awstats statistics created. Either $awstats_pl or $awstats_buildstaticpages_pl is not owned by root user.",LOGLEVEL_WARN);
        }
}

die("finished.\n");
?>
I changed a lot the script (e.g. I removed the check for the second day of month). I tried it and it works.

Can anyone confirm it?

Till, I would appreciate it a lot, if you could take a look.

The concept is:
- If this is the first day of the month, go back one day and create the old stats and copy the previous month if only, the dir (year-month) does not exist, else do nothing. (This part will execute only once, and will not execute again if the folder will be created)
- Make the date correct
- Recreate the stats for today (this will execute as many times as the cron will calls this sript)
Reply With Quote
  #5  
Old 3rd December 2011, 12:23
8omas 8omas is offline
Senior Member
 
Join Date: Jan 2011
Posts: 148
Thanks: 16
Thanked 35 Times in 26 Posts
Default

One more thing.
The command :
Code:
$command = "$awstats_buildstaticpages_pl -month='$awmonth' -year='$awyear' -update -config='$domain' -lang=".$conf['language']." -dir='$statsdir' -awstatsprog='$awstats_pl'";
does not work for my language (Greek). So I have to change -lang=".$conf['language']." with -lang=gr

I assume this happens because the awstats knows Greek as "gr" and ISPConfig as "el"
Reply With Quote
  #6  
Old 5th January 2012, 00:20
8omas 8omas is offline
Senior Member
 
Join Date: Jan 2011
Posts: 148
Thanks: 16
Thanked 35 Times in 26 Posts
Default

With the new year I had the confirmation that I wanted. The stats updated normally.
The only problem, is that I could not see the update. But the update was there!

The problem was the lines:
Code:
        if (date("d") == 1)
        {
                $month = date("m")-1;
                if (date("m") == 1)
                {
                        $year = date("Y")-1;
                        $month = "12";
                }
        }
in the file ......stats/index.php

Anyway. The stats dissappeared for only one day. When the day was 2 the stats appeared normal.

A quick fix is to comment out every instance of the above code in every index.php of every site.

Another fix (is a bad fix because we have to mess up with core) is to comment out the above lines in:

/usr/local/ispconfig/server/conf/awstats_index.php.master

I did the second. The bad thing is that I must remember this when the new update of ISPConfig will be ready.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
My Centos is terribly slow Galled Server Operation 9 7th December 2010 15:31
Vhosts...conf not synced to changes crypted General 50 24th April 2010 00:54
The system is currently updating the configuration files. warlock General 8 21st February 2009 18:15
High Availability Samba cluster - DRBD + Heartbeat djalex Server Operation 58 25th May 2007 19:38
HotSaNIC domino Tips/Tricks/Mods 23 6th November 2006 05:19


All times are GMT +2. The time now is 02:35.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.