ISPconfig 3 - awstats

[Mouton]

I quickly wrote this small plugin to create awstats config file automatically (base on the ISPconfig 2 plugin)
It really needs improvement and automation but that's a start for whoever has time to write more about it...
Do not hesitate to improve or fix it....

PS: congrats for the 3.0.1 version !


Just paste the following code in a awstats_plugin.inc.php file in the plugins-available directory.



<?php

class awstats_plugin {

var $plugin_name = 'awstats_plugin';
var $class_name = 'awstats_plugin';

//* This function is called during ispconfig installation to determine
// if a symlink shall be created for this plugin.
function onInstall() {
global $conf;

if(@$conf['services']['awstats'] == true) {
return true;
} else {
return false;
}

}


/*
This function is called when the plugin is loaded
*/

function onLoad() {
global $app;

/*
Register for the events
*/

//* Mail Domains
$app->plugins->registerEvent('web_domain_insert',$this->plugin_name,'domain_insert');
$app->plugins->registerEvent('web_domain_update',$this->plugin_name,'domain_update');
$app->plugins->registerEvent('web_domain_delete',$this->plugin_name,'domain_delete');
}


function user_insert($event_name,$data) {
// TODO: handle users for awstats access
}

function user_update($event_name,$data) {
// TODO: handle users for awstats access
}

function user_delete($event_name,$data) {
// TODO: handle users for awstats access
}

function domain_update($event_name,$data) {
global $app, $conf;

$this->domain_insert($event_name, $data);
}

function domain_insert($event_name,$data) {
global $app, $conf;

if(!@is_file("/etc/awstats/awstats.".$data["new"]["domain"].".conf") || ($data["old"]["domain"] != '' && $data["new"]["domain"] != $data["old"]["domain"])) {
if ( @is_file("/etc/awstats/awstats.".$data["old"]["domain"].".conf") )
exec("rm -f /etc/awstats/awstats.".$data["old"]["domain"].".conf");

exec( "echo 'Include \"/etc/awstats/awstats.conf\"' > /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'LogFile=\"/var/log/ispconfig/httpd/".$data["new"]["domain"]."/access.log\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'SiteDomain=\"".$data["new"]["domain"]."\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf");
// exec( "echo 'AllowAccessFromWebToFollowingAuthenticatedUsers=\"".$ht_user_list."\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf");
}

}

function domain_delete($event_name,$data) {
global $app, $conf;

if($data["old"]["domain"] != '' ) {
if ( @is_file("/etc/awstats/awstats.".$data["old"]["domain"].".conf") )
exec("rm -f /etc/awstats/awstats.".$data["old"]["domain"].".conf");
}
}




} // end class
?>

[andypl]

Ok i put this scripts in the plugins folder and how its works ?
I dont see any options on ISPConfig panel for awstats.

[till]

To enable the script you will have to create a symlink in the plugins-enabled folder to the plugin in the plugins-available folder like it is done for the other plugins. Afterwards you edit the file /usr/local/ispconfig/server/lib/config.inc.php and add the line:

$conf['services']['awstats'] = true;

Then the plugin should work. The author of the plugin may correct me if I missed an additional step as I did not tested the plugin yet. An of xourse the awstats package from your linux distribution has to be installed.

[Mouton]

Indeed, you'll need to install awstats first. Sadly, I do not have to to make any tutorial yet. I'll try but.

This plugin is only - for the moment -for generating awstats config for each ISPconfig 3 domain.

I'll work on the user htaccess restriction as soon as possible.

[archerjd]

I'll work on the user htaccess restriction as soon as possible.

Hi Mounton,
It seems to me that the .htaccess file even though has been used for decades , is a little messy. Would mod-auth-mysql be a cleaner/better solution?

One thing we have to keep in mind is if a user account gets disabled for some oddball reason. The .htaccess file will have to be modified and then again when the account is re-enabled. I believe if my memory serves me right, mod-auth-mysql has a variable just for things like that and it gives you the ability to use the same user accounts within the dbispconfig database.

There are some conns to this. mod-auth-mysql requires basic authentication (plain text).
Also, you would have to add something like this to the vhost.conf.master file.
<tmpl_if name='awstats' op='==' value='y'
<Directory <tmpl_var name='web_document_root_www'>/stats
AuthBasicAuthoritative Off
AuthUserFile /dev/null
AuthMySQL On
AuthName "AWstats Service"
AuthType Basic
Auth_MySQL_Host localhost
Auth_MySQL_User ispconfig
Auth_MySQL_Password <tmpl_var name='db_password'>
AuthMySQL_DB dbispconfig
AuthMySQL_Password_Table ftp_user
Auth_MySQL_Username_Field username
Auth_MySQL_Password_Field password
Auth_MySQL_Group_Table ftp_user
Auth_MySQL_Group_Field gid
Auth_MySQL_Encryption_Types Crypt
Auth_MySQL_Empty_Passwords Off
Auth_MySQL_Authoritative On
Auth_MySQL_Password_Clause " AND enabled=1"
require group <tmpl_var name='system_group'>
</Directory>
</tmpl_if>

[Mouton]

Good idea.

I'm not sure, but mod-auth-mysql seems to have recent security issues as well as compatibility problems with apache 2.2 (not tested).
Maybe would it be better not to use a third party module but the mod_authn_dbd (http://httpd.apache.org/docs/2.2/mod/) module ?

[till]

ISPConfig 3 uses already .htacess protection for the webalizer statistics and that is based on plain old text files for compatibility reasons and I dont see a reason to add an additional external package as requirement if it is not nescessary and if it does not bring any real benefit as ispconfig has to write all the apache config and the sattistics config as files anyway, so writing a few lines into another file do not matter.

[astewart]

I installed AWSTATS and created the plugin.
Then created a Symlink to enable the plugin.
After this I added '$conf['services']['awstats'] = true;' to the config.inc.php

I have refreshed the Sites in ISPCONFIG which created the 'awstats.demo.com.conf' but it doesn't appears to be creating any awstats folder in the Clients web folder. Nor do I see any apache config file for awstats.

Am I missing something here?

[archerjd]

Thanks Till, I stand corrected. :)
Does ISPConfig 3 re-use the existing accounts created for the site for creating the .htaccess file?

[archerjd]

Good idea.

I'm not sure, but mod-auth-mysql seems to have recent security issues as well as compatibility problems with apache 2.2 (not tested).
Maybe would it be better not to use a third party module but the mod_authn_dbd (http://httpd.apache.org/docs/2.2/mod/) module ?

Mounton,
I believe I found an article describing what you have mentioned above but it seems to be a little out dated. Do you have a link describing this as of this year?
I have been using the pre-packaged version in Debian Lenny for testing some applications and do not want to continue using it if there are still issues with it.

[till]

Thanks Till, I stand corrected. :)
Does ISPConfig 3 re-use the existing accounts created for the site for creating the .htaccess file?

There is a password input field for the statistics password in the site setting.

[Thomas Jensen]

Which username to use in the statistics?

[falko]

Try admin.

[andypl]

Ok I have a setup for each page in AWStats directory.
As now set up in cron to check all config and update data?
By default, AWStats displays http://domain.ltd/cgi-bin/awstats.pl but i want http://domain.ltd/stats

[archerjd]

I have looked into this once before and it seemed to me that mod-rewrite can do this beautifully. ;)

[letezo]

I've just installed the plugin. It creates the awstats config files for the domains, but it is far from working in its current state. Things need to do:


Cron task to update the statistics for each domain. The default cron script installed by the system uses only the awstats.conf file and hardwired to build stats from /var/log/apache/access.log only, so that is clearly not enough in our case.

Support for running awstats.pl without generally enabling CGI support for those domains. It can be protected by the same password as the Webalizer statistics. We have to provide a separate path for awstats. There should be a link from the Webalizer statistics to jump to awstats to help users.

New option for the Stats tab of the site in ISPConfig to enable/disable awstats for the domain.

Testing this on all the supported platforms.

[letezo]

I suggest moving the domain specific configuration files into a new directory like /etc/awstats/ispconfig or somewhere else. This directory should be created by the awstats plugin on installation or first use.

The Ubuntu 8.04 LTS distribution contains a script:
/usr/share/doc/awstats/awstats_updateall.pl

It can be configured (see the beginning of this file) to update the statistics for all the ISPConfig managed domains then.

Related topic, both for Webalizer and AWStats: We can enable GeoIP for them if we install the geoip-bin package and take care of downloading the free GeoIP Country database from MaxMind. I haven't tested this yet, however.

[letezo]

We have to place those config files directly under /etc/awstats, unfortunately. AWStats tries to load the config files for the virtual hosts from there.

[letezo]

We have to avoid the following bug on Ubuntu 8.04 LTS:
http://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg385558.html

Due to the above bug awstats.conf.local cannot be included from awstats.conf, since that would be a "nested" (three level deep) include. We can comment out the Include "/etc/awstats/awstats.conf.local" line at the end of awstats.conf and include that from each domain specific awstats config file. Or we can change awstats.conf directly, but that would be bad practice and might be nor so resilent to upgrades as we would want.

[vogelor]

Things need to do:


Cron task to update the statistics for each domain. The default cron script installed by the system uses only the awstats.conf file and hardwired to build stats from /var/log/apache/access.log only, so that is clearly not enough in our case.



i don't know if anybody is working on awstats support, but IF!

if you name the plugin xxx_core_module.inc.php and then create the link in the plugins-core dir instead of the plugins-enabled dir (is done automatically by the installer if the plugin is called xxx_core_module.inc.php), then the plugin is loaded and executed every minute. so you do not neet any cron taks! (have a look at the monitor_core_module i wrote some times ago. with this module i have had the same problem with the cron - job and till and i decided to implement it this way instead of creating a special cron-job for every module needed such...

[grungy]

awstats_updateall.pl on Debain is run as following:

/usr/share/doc/awstats/examples/awstats_updateall.pl now -awstatsprog=/usr/lib/cgi-bin/awstats.pl

The awstats conf file should be more rich, for example:

exec( "echo 'Include \"/etc/awstats/awstats.conf\"' > /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'LogFile=\"/var/log/ispconfig/httpd/".$data["new"]["domain"]."/access.log\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'SiteDomain=\"".$data["new"]["domain"]."\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf");
exec( "echo 'DNSLookup=\"1\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'DirData =\"/var/cache/awstats\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'AllowFullYearView=\"3\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf" );

[admins]

To the simlink: From where to where must I create a symlink ln -s ....

Thanks
admins

[djtremors]

Oo this looks to be a good plugin hack. Might add my bit to take logins from mysql and create a .htaccess htpasswd file as i'm migrating my ispc2 accounts to my ispc3 server.

long live the plugins!

Is there a list what hooks are available for plugins? I have a DR design in mind and would like to use this opportunity to mirror when accounts are created and deleted with rsync ability etc.etc.

[till]

There is no list yet. But if you look into the files in mods-available, you can easily find the definitions. Currently these are available:

client_insert, client_update, client_delete
database_insert, database_update, database_delete
mail_domain_insert, mail_domain_update, mail_domain_delete
mail_user_insert, mail_user_update, mail_user_delete
mail_access_insert, mail_access_update, mail_access_delete
mail_forwarding_insert, mail_forwarding_update, mail_forwarding_delete
mail_transport_insert, mail_transport_update, mail_transport_delete
mail_get_insert, mail_get_update, mail_get_delete
mail_content_filter_insert, mail_content_filter_update, mail_content_filter_delete
server_insert, server_update, server_delete
server_ip_insert, server_ip_update, server_ip_delete
firewall_insert, firewall_update, firewall_delete
software_update_inst_insert, software_update_inst_update, software_update_inst_delete
web_domain_insert, web_domain_update, web_domain_delete
ftp_user_insert, ftp_user_update, ftp_user_delete
shell_user_insert, shell_user_update, shell_user_delete

The event names correspond with the database table names.

[BorderAmigos]

I did all of the above and still not working. I'll read it thru 5 or 10 more times to find what I'm missing.

[djtremors]

@BorderAmigos : What does "still not working" mean. Did you make a vhost for it

I had to change mine to work with debians apt-get version which was different.


if(!@is_file("/etc/awstats/awstats.".$data["new"]["domain"].".conf") || ($data["old"]["domain"] != '' && $data["new"]["domain"] != $data["old"]["domain"])) {
if ( @is_file("/etc/awstats/awstats.".$data["old"]["domain"].".conf") )
exec("rm -f /etc/awstats/awstats.".$data["old"]["domain"].".conf");

exec( "echo 'Include \"/etc/awstats/awstats.conf\"' > /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'LogFile=\"/var/log/ispconfig/httpd/".$data["new"]["domain"]."/access.log\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'SiteDomain=\"".$data["new"]["domain"]."\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf");
// exec( "echo 'AllowAccessFromWebToFollowingAuthenticatedUsers=\"".$ht_user_list."\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf");
}


my /etc/apache2/conf.d/awstats.conf looks like this (for now).
Alias /awstatsclasses "/usr/share/awstats/classes/"
Alias /awstatscss "/usr/share/awstats/css/"
Alias /awstats-icon "/usr/share/awstats/icon/"
ScriptAlias /stats/ "/usr/lib/cgi-bin/"
#
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/share/awstats">
Options None
AllowOverride None
Order allow,deny
Allow from all
</Directory>

what I did notice is that this only supports the domain and not the www. autodomains which I'll get around to adding along with authentication (been sick lately any contenders to SQL to get list, be my guest)...

[djtremors]

Ok, got around to making a quick patch to allow my awstats to appear using the configuration method mentioned earlier and using a redirection from /stats to redirect to /ispcstats with the domain.

This will create a .htaccess_stats file with the clients login/password from the database. Change as you please if you want email users to be able to use it instead etc. There is no super admin login atm but that's an easy addition by adding a admin:{hash} to the write routine.

There may be other little tweaks needs in regards to getting awstats part working but this is only the ispc3 side.

Code :
Ok, got around to making a quick patch to allow my awstats to appear using the configuration method mentioned earlier and using a redirection from /stats to redirect to /ispcstats with the domain.

This will create a .htaccess_stats file with the clients login/password from the database. Change as you please if you want email users to be able to use it instead etc. There is no super admin login atm but that's an easy addition by adding a admin:{hash} to the write routine.

There may be other little tweaks needs in regards to getting awstats part working but this is only the ispc3 side.

Code :
[PHP]<?php

class awstats_plugin {

var $plugin_name = 'awstats_plugin';
var $class_name = 'awstats_plugin';

//* This function is called during ispconfig installation to determine
// if a symlink shall be created for this plugin.
function onInstall() {
global $conf;

if(@$conf['services']['awstats'] == true) {
return true;
} else {
return false;
}

}


/*
This function is called when the plugin is loaded
*/

function onLoad() {
global $app;

/*
Register for the events
*/

//* Mail Domains
$app->plugins->registerEvent('web_domain_insert',$this->plugin_name,'domain_insert');
$app->plugins->registerEvent('web_domain_update',$this->plugin_name,'domain_update');
$app->plugins->registerEvent('web_domain_delete',$this->plugin_name,'domain_delete');
$app->plugins->registerEvent('client_insert',$this->plugin_name,'client_insert');
$app->plugins->registerEvent('client_update',$this->plugin_name,'client_update');
$app->plugins->registerEvent('client_delete',$this->plugin_name,'client_delete');
}

function awstats_update($event_name,$data) {
global $app, $conf;
$result = $app->db->queryAllRecords("SELECT * FROM web_domain wd,sys_group sg,client c where wd.sys_groupid = sg.groupid and sg.client_id = c.client_id");
$fp = fopen($domroot."/.htpasswd_stats","w");
if ($fp)
{
foreach($result as $row)
{
$domain = $row['domain'];
$username = $row['username'];
$password = $row['password'];
$domroot = $row['document_root'];
echo "Domain::$domain DomRoot::$domroot\n";

fwrite($fp,$username.":".$password."\n");
}
fclose($fp);
}
$fp = fopen($domroot."/web/stats/.htaccess","w");
if ($fp)
{
$access='AuthType Basic
AuthName "Site Client Access Only"
AuthUserFile '.$domroot.'/.htpasswd_stats
<limit GET PUT POST>
require valid-user
</limit>
Redirect 301 /stats http://www.'.$domain.'/ispcstats/awstats.pl?config=www.'.$domain.'
';

fwrite($fp,$access);
fclose($fp);
}
}

function client_insert($event_name,$data) {
global $app, $conf;
$this->awstats_update();
}

function client_update($event_name,$data) {
global $app, $conf;
$this->awstats_update();
}

function client_delete($event_name,$data) {
global $app, $conf;
$this->awstats_update();
}

function user_insert($event_name,$data) {
// TODO: handle users for awstats access
}

function user_update($event_name,$data) {
// TODO: handle users for awstats access
}

function user_delete($event_name,$data) {
// TODO: handle users for awstats access
}

function domain_update($event_name,$data) {
global $app, $conf;

$this->domain_insert($event_name, $data);
}

function domain_insert($event_name,$data) {
global $app, $conf;

if(!@is_file("/etc/awstats/awstats.".$data["new"]["domain"].".conf") || ($data["old"]["domain"] != '' && $data["new"]["domain"] != $data["old"]["domain"])) {
if ( @is_file("/etc/awstats/awstats.".$data["old"]["domain"].".conf") )
exec("rm -f /etc/awstats/awstats.".$data["old"]["domain"].".conf");

exec( "echo 'Include \"/etc/awstats/awstats.conf\"' > /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'LogFile=\"/var/log/ispconfig/httpd/".$data["new"]["domain"]."/access.log\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf" );
exec( "echo 'SiteDomain=\"".$data["new"]["domain"]."\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf");
// exec( "echo 'AllowAccessFromWebToFollowingAuthenticatedUsers=\"".$ht_user_list."\"' >> /etc/awstats/awstats.".$data["new"]["domain"].".conf");

}

}

function domain_delete($event_name,$data) {
global $app, $conf;

if($data["old"]["domain"] != '' ) {
if ( @is_file("/etc/awstats/awstats.".$data["old"]["domain"].".conf") )
exec("rm -f /etc/awstats/awstats.".$data["old"]["domain"].".conf");
}
}




} // end class
?>