Extending Perfect Server - Debian Squeeze [ISPConfig 3] - Page 4
On this page
9. Anytime stats with awstats
In this section, you can apply a modification that will allow you to update the awstats anytime you want, or (by using cron) in more frequent time intervals. Before we proceed we must change the default log format for awstats. In Debian Squeeze the default format is 4, but we need the 1. So either you edit the file /etc/awstats/awstats.conf or the file /etc/awstats/awstats.conf.local . I prefer the second one, because it will let me keep up with updates:
nano /etc/awstats/awstats.conf.local
and append the line: LogFormat=1
After that, the file should look like:
# You can overrides config directives here.
# This is particularly useful for users with several configs for
# different virtual servers, who want to reuse common parameters.
# Also, this file is not updated with each new upstream release.
LogFormat=1
Now we have to copy the default daily cron of ISPConfig and edit it ... actually we have to edit it A LOT (new file name : mycron.php ). Basically we have to cutoff a lot of lines and only keep some of them. Do:
cp /usr/local/ispconfig/server/cron_daily.php /root/scripts/mycron.php
chmod 700 /root/scripts/mycron.php
nano /root/scripts/mycron.php
and paste all the following lines:
<?php
/*
Copyright (c) 2007, Till Brehm, projektfarm Gmbh
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of ISPConfig nor the names of its contributors
may be used to endorse or promote products derived from this software without
specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
require('/usr/local/ispconfig/server/lib/config.inc.php');
require('/usr/local/ispconfig/server/lib/app.inc.php');
set_time_limit(0);
// make sure server_id is always an int
$conf['server_id'] = intval($conf['server_id']);
// Load required base-classes
$app->uses('ini_parser,file,services,getconf');
#######################################################################################################
// Create awstats statistics
#######################################################################################################
$sql = "SELECT domain_id, domain, document_root FROM web_domain WHERE stats_type = 'awstats' AND server_id = ".$conf['server_id'];
$records = $app->db->queryAllRecords($sql);
$web_config = $app->getconf->get_server_config($conf['server_id'], 'web');
foreach($records as $rec) {
$logfile = escapeshellcmd($rec['document_root'].'/log/access.log');
$domain = escapeshellcmd($rec['domain']);
$statsdir = escapeshellcmd($rec['document_root'].'/web/stats');
$awstats_pl = $web_config['awstats_pl'];
$awstats_buildstaticpages_pl = $web_config['awstats_buildstaticpages_pl'];
$awstats_conf_dir = $web_config['awstats_conf_dir'];
$awstats_website_conf_file = $web_config['awstats_conf_dir'].'/awstats.'.$domain.'.conf';
if(is_file($awstats_website_conf_file)) unlink($awstats_website_conf_file);
if(!is_file($awstats_website_conf_file)) {
$awstats_conf_file_content = 'Include "'.$awstats_conf_dir.'/awstats.conf"
LogFile="/var/log/ispconfig/httpd/'.$domain.'/access.log"
SiteDomain="'.$domain.'"
HostAliases="www.'.$domain.' localhost 127.0.0.1"';
file_put_contents($awstats_website_conf_file,$awstats_conf_file_content);
}
if(!@is_dir($statsdir)) mkdir($statsdir);
// awstats_buildstaticpages.pl -update -config=mydomain.com -lang=en -dir=/var/www/domain.com/web/stats -awstatsprog=/path/to/awstats.pl
$command = "$awstats_buildstaticpages_pl -update -config='$domain' -lang=en -dir='$statsdir' -awstatsprog='$awstats_pl'";
if($awstats_pl != '' && $awstats_buildstaticpages_pl != '' && fileowner($awstats_pl) == 0 && fileowner($awstats_buildstaticpages_pl) == 0) {
exec($command);
rename($rec['document_root'].'/web/stats/awstats.'.$domain.'.html',$rec['document_root'].'/web/stats/index.html');
$app->log('Created awstats statistics with command: '.$command,LOGLEVEL_DEBUG);
} else {
$app->log("No awstats statistics created. Either $awstats_pl or $awstats_buildstaticpages_pl is not owned by root user.",LOGLEVEL_WARN);
}
}
die("finished.\n");
?>
This script is a cutoff of the ISPConfig's default cron file and applies up to ISPConfi 3.0.3. It's made by Till Brehm , and we edited it, so as the update procedure DOES NOT modify any system (or log) file whenever we call it. In simple words, the script reads the access.log file of each domain and DOES NOT do any rotation or any modification, except that it reproduces the html files for awstats. Keep in mind that the default ISPConfig cron job, will run as usual at 00:30 and it will simply append to the stats db and recreate the html files.
In the above script you can change the language produced by awstats if you simply change the letters after "lang=" with those that represent your country.
For ISPConfig 3.0.4 please use the following code:
<?php
/*
Copyright (c) 2007, Till Brehm, projektfarm Gmbh
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of ISPConfig nor the names of its contributors
may be used to endorse or promote products derived from this software without
specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
require('/usr/local/ispconfig/server/lib/config.inc.php');
require('/usr/local/ispconfig/server/lib/app.inc.php');
set_time_limit(0);
// make sure server_id is always an int
$conf['server_id'] = intval($conf['server_id']);
// Load required base-classes
$app->uses('ini_parser,file,services,getconf');
#######################################################################################################
// Create awstats statistics
#######################################################################################################
$sql = "SELECT domain_id, domain, document_root FROM web_domain WHERE stats_type = 'awstats' AND server_id = ".$conf['server_id'];
$records = $app->db->queryAllRecords($sql);
$web_config = $app->getconf->get_server_config($conf['server_id'], 'web');
foreach($records as $rec) {
$logfile = escapeshellcmd($rec['document_root'].'/log/access.log');
$domain = escapeshellcmd($rec['domain']);
$statsdir = escapeshellcmd($rec['document_root'].'/web/stats');
$awstats_pl = $web_config['awstats_pl'];
$awstats_buildstaticpages_pl = $web_config['awstats_buildstaticpages_pl'];
$awstats_conf_dir = $web_config['awstats_conf_dir'];
$awstats_website_conf_file = $web_config['awstats_conf_dir'].'/awstats.'.$domain.'.conf';
if(is_file($awstats_website_conf_file)) unlink($awstats_website_conf_file);
$sql = "SELECT domain FROM web_domain WHERE (type = 'alias' OR AND type = 'subdomain') server_id = ".$conf['server_id'];
$aliases = $app->db->queryAllRecords($sql);
$aliasdomain = '';
if(is_array($aliases)) {
foreach ($aliases as $alias) {
$aliasdomain.= ' '.$alias['domain']. ' www.'.$alias['domain'];
}
}
if(!is_file($awstats_website_conf_file)) {
$awstats_conf_file_content = 'Include "'.$awstats_conf_dir.'/awstats.conf"
LogFile="/var/log/ispconfig/httpd/'.$domain.'/access.log"
SiteDomain="'.$domain.'"
HostAliases="www.'.$domain.' localhost 127.0.0.1"'.$aliasdomain;
file_put_contents($awstats_website_conf_file,$awstats_conf_file_content);
}
if(!@is_dir($statsdir)) mkdir($statsdir);
####################################################################################
######################### RECREATE STATS ###########################################
$redo = 0; // set this to 1 if you want to recreate statistics for a specific month
$REMONTH = 11; // select the month
$REYEAR = 2011; //select the year
$LANG = 'gr'; // set your language
// After setting the above rerun this script
// e.g
// php /path/to
####################################################################################
####################################################################################
if ( $redo == 1 ) {
$awmonth = $REMONTH;
$awyear = $REYEAR;
$statsdirold = $statsdir."/".$awyear."-".$awmonth."/";
if (!is_dir($statsdirold)) {
mkdir($statsdirold);
}
$command = "$awstats_buildstaticpages_pl -month='$awmonth' -year='$awyear' -update -config='$domain' -lang='$LANG' -dir='$statsdirold' -awstatsprog='$awstats_pl'";
if($awstats_pl != '' && $awstats_buildstaticpages_pl != '' && fileowner($awstats_pl) == 0 && fileowner($awstats_buildstaticpages_pl) == 0) {
exec($command);
rename($statsdirold.'awstats.'.$domain.'.html',$statsdirold.'awsindex.html');
$app->log('Created awstats statistics with command: '.$command,LOGLEVEL_DEBUG);
} else {
$app->log("No awstats statistics created. Either $awstats_pl or $awstats_buildstaticpages_pl is not owned by root user.",LOGLEVEL_WARN);
}
}
else {
if (date("d") == 1) {
$awmonth = date("n")-1;
$awyear = date("Y");
if (date("n") == 0) {
$awyear = date("Y")-1;
$awmonth = "12";
}
$statsdirold = $statsdir."/".$awyear."-".$awmonth."/";
if (!is_dir($statsdirold)) {
mkdir($statsdirold);
$command = "$awstats_buildstaticpages_pl -month='$awmonth' -year='$awyear' -update -config='$domain' -lang='$LANG' -dir='$statsdirold' -awstatsprog='$awstats_pl'";
if($awstats_pl != '' && $awstats_buildstaticpages_pl != '' && fileowner($awstats_pl) == 0 && fileowner($awstats_buildstaticpages_pl) == 0) {
exec($command);
rename($statsdirold.'awstats.'.$domain.'.html',$statsdirold.'awsindex.html');
$app->log('Created awstats statistics with command: '.$command,LOGLEVEL_DEBUG);
} else {
$app->log("No awstats statistics created. Either $awstats_pl or $awstats_buildstaticpages_pl is not owned by root user.",LOGLEVEL_WARN);
}
}
}
// ispconfig, on the second day of month, overwrites the stats created before by this script
// to overwrite ispconfig default execution on 2nd day of month we must execute the below code
if (date("d") == 2) {
$awmonth = date("n")-1;
$awyear = date("Y");
if (date("n") == 0) {
$awyear = date("Y")-1;
$awmonth = "12";
}
$statsdirold = $statsdir."/".$awyear."-".$awmonth."/";
$command = "$awstats_buildstaticpages_pl -month='$awmonth' -year='$awyear' -update -config='$domain' -lang='$LANG' -dir='$statsdirold' -awstatsprog='$awstats_pl'";
if($awstats_pl != '' && $awstats_buildstaticpages_pl != '' && fileowner($awstats_pl) == 0 && fileowner($awstats_buildstaticpages_pl) == 0) {
exec($command);
rename($statsdirold.'awstats.'.$domain.'.html',$statsdirold.'awsindex.html');
$app->log('Created awstats statistics with command: '.$command,LOGLEVEL_DEBUG);
} else {
$app->log("No awstats statistics created. Either $awstats_pl or $awstats_buildstaticpages_pl is not owned by root user.",LOGLEVEL_WARN);
}
}
// end of overwrite
// unfortunately the above will execute many times (as many as you run this script by cron)
$awmonth = date("n");
$awyear = date("Y");
$command = "$awstats_buildstaticpages_pl -month='$awmonth' -year='$awyear' -update -config='$domain' -lang='$LANG' -dir='$statsdir' -awstatsprog='$awstats_pl'";
if($awstats_pl != '' && $awstats_buildstaticpages_pl != '' && fileowner($awstats_pl) == 0 && fileowner($awstats_buildstaticpages_pl) == 0) {
exec($command);
if(is_file($rec['document_root'].'/web/stats/index.html')) unlink($rec['document_root'].'/web/stats/index.html');
rename($rec['document_root'].'/web/stats/awstats.'.$domain.'.html',$rec['document_root'].'/web/stats/awsindex.html');
if(!is_file($rec['document_root']."/web/stats/index.php")) copy("/usr/local/ispconfig/server/conf/awstats_index.php.master",$rec['document_root']."/web/stats/index.php");
$app->log('Created awstats statistics with command: '.$command,LOGLEVEL_DEBUG);
} else {
$app->log("No awstats statistics created. Either $awstats_pl or $awstats_buildstaticpages_pl is not owned by root user.",LOGLEVEL_WARN);
}
} //end redo
} // end foreach
die("finished.\n");
?>
Keep in mind (in version > =3.0.4) that you will not see any stats the first day of every
month (as with the normal script). If you want to see the udated stats
after the first execution the first day of the month you have to read
this post : https://www.howtoforge.com/forums/showthread.php?p=270816#post270816
To execute the script every 4 hours, first create the file /root/scripts/mycron.sh, make it executable and then add it to the cronjob list:
touch /root/scripts/mycron.sh
chmod 700 /root/scripts/mycron.sh
nano /root/scripts/mycron.sh
and paste the following:
#!/bin/sh
PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin
/usr/bin/php /root/scripts/mycron.php
Add the cronjob:
crontab -e
Append the following line, save and exit:
45 0,4,8,12,16,20 * * * /root/scripts/mycron.sh
You can also initiate the stats creation anytime by:
/root/scripts/mycron.sh
You can recreate stats of previously months (as long as there are data in /var/lib/awstats or /var/cache/awstats). Read the comments in the script, set redo=1 and execute the script. More at https://www.howtoforge.com/forums/showthread.php?t=51731
10. mysql tuning
Download tuning-primer.sh and mysqltuner.pl. These will help you to improve mysql's configuration file.
cd /root/scripts
wget http://www.day32.com/MySQL/tuning-primer.sh
wget http://mysqltuner.com/mysqltuner.pl
chmod 700 tuning-primer.sh mysqltuner.pl
To execute them:
perl /root/scripts/mysqltuner.pl
/root/scripts/tuning-primer.sh
The scripts will ask some simple questions (user/pass) and will display in red colour, their suggestions for critical settings. You can use their suggestions to boost the performance of mysql-server.
