use jawstats / mawstats with ispconfig

[e100]

Its 4:45AM so blame any bugs on lack of sleep

The script is designed to be called by ispconfig cron as if it was the awstats_buildstaticpages.pl. No changes are required to ispconfig other than editing the awstats_buildstaticpages.pl setting in the panel.

This setup works with ispconfig 3.0.3, just follow the directions that are documented in the script and you should have a painless transition from static awstats to jawstats goodness. This is assuming you have awstats setup and working with ispconfig 3 already.

Enjoy!

#!/bin/bash

#Location of your jawstats:
JAWSTATS=/usr/local/ispcjawstats/jawstats/

#Location of your awstats config files, should match setting in ispconfig 3
AWSTATSCONFIG=/etc/awstats

#Location of old awstats data directory
OLDDIR=/var/lib/awstats/

#Optional Setting
DANGEROUS="no"
# Setting this to "YES" allows the script to delete the old static awstats html files.
#
# This is the only option that should result in anything being deleted
# However, as with any script use this option and this script at your own risk.
# No warranties implied, infact this script may delete all of your data 
# and make your hair fall out.
#
# Always make a backup and I hope you find this script useful
# 
##
## Do not edit below this line
##
#####################################################################################
## Many thanks to everyone who has contributed to ispconfig
## -e100
#####################################################################################
# SETUP
#
# 1. Create a directory to hold this script
#    I placed my script in /usr/local/ispcjawstats
#    ie mkdir /usr/local/ispcjawstats   
# 2. Put this script in that directory
# 3. Make this script executible
#    ie chmod 555 /usr/local/ispcjawstats/ispcjawstats.sh
# 4. Create your jawstats "source" folder. This is the folder that this script
#    will copy from to install jawstats for each site that is configured for awstats
#    in ispconfig 3
#    ie mkdir /usr/local/ispcjawstats/jawstats
# 5. Put jawstats into this folder
# 6. Set the JAWSTATS variable above to match the location of your jawstats folder
# 7. Set the AWSTATS variable above to match the location if your awstats config folder
#       This should match what you have set in ispconfig 3
# 8. delete the config.dist.php in the jawstats folder
# 9. create a file named "version" in the jawstats folder.
#    If the contents of this file is changed
#    this script will update jawstats in your customers stats folder
#    I put "0.7" inside of version to denote the version.
#    The contents of this file do not matter, changing the contents triggers the update.
# 10.Create a config.php in the jawstats directory that looks like this:
:<<StartCopyNextLineStopBeforeYouSeeThisAgain
<?php

  // core config parameters
  $sDefaultLanguage      = "en-gb";
  $sConfigDefaultView    = "thismonth.all";
  $bConfigChangeSites    = false;
  $bConfigUpdateSites    = false;
  $sUpdateSiteFilename   = "xml_update.php";

  // individual site configuration
  $aConfig["<DOMAIN>"] = array(
    "statspath"   => "<PATH>",
    "statsname"   => "myformat[MM][YYYY].txt",
    "updatepath"  => "<AWSTATS>",
    "siteurl"     => "http://<DOMAIN>",
    "sitename"    => "<DOMAIN>",
    "theme"       => "default",
    "fadespeed"   => 250,
    "password"    => "",
    "includes"    => "",
    "language"    => "en-gb"
  );

?>
StartCopyNextLineStopBeforeYouSeeThisAgain
#
# NOTE: if you are using mawstats you may need to add more options to the above configuration
# 11. Set the permissions on every file/folder inside the jawstats directory
#     to whatever you feel is appropriate for your server configuration.
#     These will be preserved when it copies.
#     I suggest the following:
#     everything  chown root:root
#     All directories chmod 555
#     All files chmod 444 except for config.php I made 644
#     This way your customers sites can read them but not change them.
# 12. In ispconfig 3 control panel go to System->Server Config
# 13. Select the server you installed this onto
# 14. Go to the web tab and change "awstats_buildstaticpages.pl script"
#     To the path to this script.
# 15. Make note of previous setting for "awstats data folder"
#     Set the variable OLDDIR to the current "awstats data folder" setting. 
# 16. Set the "awstats data folder" to [website_path]/web/stats/  
#
# Wait for the daily cron to run or run it yourself:
# /usr/local/ispconfig/server/cron_daily.sh
#
# point your browser to http://sitename/stats/ and log in as usual
#
# Happy JAWStats!
#
# NOTES:
# If you had awstats setup prior to installing this script and you
# setup the OLDDIR variable correctly this script will copy the
# previous awstats DB files for the site into the new 
# [website_path]/web/stats/ directory.
#
# If your awstats files are defined different than awstats[MM][YYYY]sitename.conf
# This script is likely to not work for you.
# The above example is how my debian ispconfig3 install is setup by default.
# Therefore this is how I have written the script to function.
# If your setup is different you will need to edit this script to make it work.
#
# If this jawstats config.php is ever missing, this script will re-create it
#
if [ ! -r "$JAWSTATS""/version" ]
then
  echo "Your JAWSTATS directory is missing or not configured properly"
  exit 1
fi

for i in $*
do
    case $i in
        -config=*)
        CONFIG=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
        ;;
        -lang=*)
        LANG=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
        ;;
        -dir=*)
        DIR=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
        ;;
    -awstatsprog=*)
        AWSTATS=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
                ;;
      esac
done

#Check if awstats config exists
if [ ! -r "$AWSTATSCONFIG""/awstats.""$CONFIG"".conf" ]
then
  echo "Your config file is missing!"
  exit 1
fi

#check if directory exists
if [ ! -r "$DIR" ]
then
  echo "Your destination directory is missing!"
  exit 1
fi

updateconfig()
{
#use sed to updateconfig
sed "s,<AWSTATS>,$AWSTATS,g" "$JAWSTATS""/config.php" \
|sed "s,<PATH>,$DIR/,g"|sed "s,<DOMAIN>,$CONFIG,g" >"$DIR""/config.php"
}

#Check if this script has ever installed JAWStats
if [ ! -r "$DIR""/version" ]
then
  #Attempt to copy the old awstats data files to new directory
  cp -pu "$OLDDIR""awstats"*"$CONFIG"".txt" "$OLDDIR""dnscachelastupdate.""$CONFIG"".hash" "$DIR"
  #Call function to update config here
  updateconfig
fi

#Check is jawstats needs updated in the destination directory
cmp "$JAWSTATS""/version" "$DIR""/version" &> /dev/null
if [ $? -eq 0 ]         # Test exit status of "cmp" command.
then
  #Jawstats is up to date
  #Check for config, if missing re-create it
  if [ ! -r "$DIR""/config.php" ]
  then
    #Call function to update config
    updateconfig
  fi 
else  
  #Jawstats is out of date, update it in user's directory.
  cp -Rpu "$JAWSTATS""/"* "$DIR""/"
  updateconfig
fi

#now lets update the stats!
#That is the whole purpose of this script, right?!

#Needed to keep perl from spitting out warnings on my system
LANG="en_US.UTF-8"
CMD="$AWSTATS""  -config='""$CONFIG""' -lang=""$LANG"" -update"
$CMD

#Now that JAWStats is installed lets create that html file that ispconfig 3 will want to rename

cat > "$DIR""/awstats.""$CONFIG"".html" <<index-html-output
<html>
<head>
<meta http-equiv="refresh" content="0;URL=index.php" />
<title>Page has moved</title>
</head>
<body>
If this page does not redirect automatically click <a href="index.php">here</a>
</body>
</html>
index-html-output



if [ "$DANGEROUS" == "YES" ]
then
  NAMES="allhosts.html allrobots.html browserdetail.html errors404.html keyphrases.html keywords.html lasthosts.html lastrobots.html osdetail.html refererpages.html refererse.html session.html unknownbrowser.html unknownip.html unknownos.html urldetail.html urlentry.html urlexit.html alldomains.html"
fi
  for ext in $NAMES
  do
    rm -f "$DIR""/awstats.""$CONFIG"".$ext"
  done
exit 0

[linus3x]

This worked great! Thanks a ton!

Here are my notes picking up from Step 11:

# 11. Set the permissions on every file/folder inside the jawstats directory
# to whatever you feel is appropriate for your server configuration.
# These will be preserved when it copies.
# I suggest the following:
# everything chown root:root
# All directories chmod 555
# All files chmod 444 except for config.php I made 644
# This way your customers sites can read them but not change them.

Here’s what I did:

Copy the jawstats folder over to the server using FTP. I dropped the jawstats folder into the client1/web1 directory.

Use the cp –r command to copy the directory down to /usr/local/ispcjawstats/:

cp -r /var/www/clients/client1/web1/jawstats /usr/local/ispcjawstats/jawstats

# everything chown root:root

chown -R root:root /usr/local/ispcjawstats/jawstats

# All directories chmod 555

find /usr/local/ispcjawstats/jawstats -type d -exec chmod 555 {} \;

# All files chmod 444 except for config.php I made 644

find /usr/local/ispcjawstats/jawstats -type f -exec chmod 444 {} \;
chmod 644 /usr/local/ispcjawstats/jawstats/config.php

When you’re done, remember to delete jawstats from the client's website.

rm -r /var/www/clients/client1/web1/jawstats

To run it on just one site (adjust the bolded items as needed):

/usr/local/ispcjawstats/ispcjawstats.sh -config=yourdomain.com -update -lang=en -awstatsprog=/usr/lib/cgi-bin/awstats.pl -dir=/var/www/clients/client1/web1/web/stats

Rename the main file:

cp /var/www/clients/client1/web1/web/stats/awstats.yourdomain.com.html /var/www/clients/client1/web1/web/stats/index.html

Then to see it:

http://www.yourdomain.com/stats/index.php

[linus3x]

OK - what we discovered on our system was: This script ran great the first time then after that, it never updated with new data. We kept seeing stats from the initial run of the script, but no updates.

After debugging a bit, I found that this script wasn't copying over the two awstats files once awstats.pl had finished building them.

I added the following line to the script and things are updating as they should:

[...]
#Needed to keep perl from spitting out warnings on my system
LANG="en_US.UTF-8"
CMD="$AWSTATS""  -config='""$CONFIG""' -lang=""$LANG"" -update"
$CMD

#NEW LINE
#Copy over the stat files that awstats just generated
cp -pu "$OLDDIR""awstats"*"$CONFIG"".txt" "$OLDDIR""dnscachelastupdate.""$CONFIG"".hash" "$DIR"

#Now that JAWStats is installed lets create that html file that ispconfig 3 will want to rename

cat > "$DIR""/awstats.""$CONFIG"".html" <<index-html-output
<html>
<head>
[...]

Make sure your OLDDIR option at the top of the script points to the folder where awstats.pl writes its output and you should be good to go. The default OLDDIR worked on our system.

This worked for us - your mileage may vary!

[triscan]

awesome work, thanks a lot.

there is a problem in ispconfig so that awstats only create a field from 0 - 0:30
http://bugtracker.ispconfig.org/inde...s&task_id=1395

this is fixed in ispconfig 3.0.3.1
to fix this in jawstats add/change some lines to the script:

[...]
for i in $*
do
    case $i in
        -config=*)
        CONFIG=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
        ;;
        -lang=*)
        LANG=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
        ;;
        -dir=*)
        DIR=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
        ;;
    -awstatsprog=*)
        AWSTATS=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
                ;;
        -LogFile=*)
        LOGFILE=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
        ;;
      esac
done
[...]

[...]
#Needed to keep perl from spitting out warnings on my system
LANG="en_US.UTF-8"
CMD="$AWSTATS""  -config='""$CONFIG""' -lang=""$LANG"" -LogFile=""$LOGFILE"" -update"
$CMD
[...]