ISPconfig is not processing apache logs: the web Traffic overview and stats are gone

[KingBeto]

Hi everyone!
I have a very big problem.. My ISPconfig is not processing the Apache Logs.
Therefore the system does not have any web statistics and the Traffic overview is wrong since it is only processing ftp and email traffic.

Here I have an example. Note that the web traffic is not processed, but I have the same amount of visits in Google Analytics each month:

Month, Web, FTP, Mail, Overall
11/2009:, 0.00, 9.65, 29.74, 39.38
10/2009:, 0.00, 26.41, 33.20, 59.61
09/2009:, 9756.34, 21.90, 40.51, 9818.74
08/2009:, 428163.42, 1999.26, 69.78, 430232.46


I checked the apache httpd.conf and everything is ok... And I checked the web.log file that the apache generates and it is OK too.

Since the ISPconfig is not processing the web.log file, it does not delete old logs after the rotation, so old logs are accumulating and using all the disc space..

Does anyone had the same problem? Any Idea of what should I do?? I'm lost and it is a very serious problem for me! Thank you!!

[till]

Please post the output of:

crontab -l

[KingBeto]

# crontab -l
1 1 * * * /scripts/snapshot-adverit.sh
1 2 * * * /scripts/snapshot-mysql.sh
0 4 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/awstats.php &> /dev/null
30 00 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/logs.php &> /dev/null
59 23 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/ftp_logs.php &> /dev/null
59 23 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/mail_logs.php &> /dev/null
59 23 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/cleanup.php &> /dev/null
0 4 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/webalizer.php &> /dev/null
0,30 * * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/check_services.php &> /dev/null
15 3,15 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/quota_msg.php &> /dev/null
40 00 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/traffic.php &> /dev/null
05 02 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/backup.php &> /dev/null
41 5 * * * /etc/webmin/cron/tempdelete.pl

Thank you!

[till]

Please run this command as root:

/root/ispconfig/php/php /root/ispconfig/scripts/shell/logs.php

and when it is finished, run:

/root/ispconfig/php/php /root/ispconfig/scripts/shell/webalizer.php

Do you get any errors?

[KingBeto]

Here it is. Please Note that for some reason, the ispconfig creates junk folders in the /var/www folder.
Like the folder "/var/www/foPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; OfficeLiveConnector.1.3;fficeLivePatch.0.0)" without quotes.



root@azul:~# /root/ispconfig/php/php /root/ispconfig/scripts/shell/logs.php

Warning: readlink(): Invalid argument in /root/ispconfig/scripts/shell/logs.php on line 125

Warning: readlink(): Invalid argument in /root/ispconfig/scripts/shell/logs.php on line 125
sh: -c: line 0: syntax error near unexpected token `)'
sh: -c: line 0: `chown -R 0:web0 /var/www/foPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; OfficeLiveConnector.1.3; OfficeLivePatch.0.0)"'



root@azul:~# /root/ispconfig/php/php /root/ispconfig/scripts/shell/webalizer.php
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
[...]

Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
15194 records (15194 ignored) in 3.00 seconds, 5064/sec
359872 records (359872 ignored) in 76.50 seconds, 4704/sec
2182 records (2182 ignored) in 0.00 seconds
57468789 records (57468789 ignored) in 94860.50 seconds, 605/sec
root@azul:~#

[till]

There is strange directory in /var/www/ which causes the script to stop. Please take a look in /var/www/, there must be a directory with the name "foPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; OfficeLiveConnector.1.3; OfficeLivePatch.0.0)". Delete that directory and run the scripts again.

[KingBeto]

Till:
I deleted junk folders. I ran the same command once again as root (root@azul:~# /root/ispconfig/php/php /root/ispconfig/scripts/shell/logs.php) and I discovered that the logs.php script was generating those junk folders.
This time it generated a 'ET CLR 3.0.4506.2152; .NET CLR 3.5.30' folder and a localhost folder. I deleted them and ran the same command once again. And the script created AGAIN other junk folders. What do you think It is going on?? Very strange!

Do you think that maybe the Apache is generating a uncompatible with ISPConfig log file?

Thank you in advance!!!

[falko]

Can you post an excerpt of your Apache access log?

[KingBeto]

Hi!
This is an example of the log generated. I attached my apache2.conf file.
Something strange is that it has several log config lines. Weird..

This is the output of the "tail -f web.log" command:

190.30.35.199 - - [03/Dec/2009:23:59:59 -0300] "GET /images/compartir/linkedin.gif HTTP/1.1" 200 746 "http://www.cpca.org.ar/contenido/a420_Flan-de-manzana" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6.3; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 1.1.4322; .NET CLR 3.0.30729)"
190.30.35.199 - - [03/Dec/2009:23:59:59 -0300] "GET /images/compartir/linkedin.gif HTTP/1.1" 200 746 "http://www.cpca.org.ar/contenido/a420_Flan-de-manzana" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6.3; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 1.1.4322; .NET CLR 3.0.30729)"
190.30.35.199 - - [03/Dec/2009:23:59:59 -0300] "GET /images/perfil/ico_enviar_pm.png HTTP/1.1" 200 1686 "http://www.cpca.org.ar/contenido/a420_Flan-de-manzana" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6.3; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 1.1.4322; .NET CLR 3.0.30729)"
190.30.35.199 - - [03/Dec/2009:23:59:59 -0300] "GET /images/perfil/ico_enviar_pm.png HTTP/1.1" 200 1686 "http://www.cpca.org.ar/contenido/a420_Flan-de-manzana" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6.3; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 1.1.4322; .NET CLR 3.0.30729)"
190.30.35.199 - - [03/Dec/2009:23:59:59 -0300] "GET /images/common/bt_ingresar.gif HTTP/1.1" 200 543 "http://www.cpca.org.ar/contenido/a420_Flan-de-manzana" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6.3; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 1.1.4322; .NET CLR 3.0.30729)"
190.30.35.199 - - [03/Dec/2009:23:59:59 -0300] "GET /images/common/bt_ingresar.gif HTTP/1.1" 200 543 "http://www.cpca.org.ar/contenido/a420_Flan-de-manzana" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6.3; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 1.1.4322; .NET CLR 3.0.30729)"
190.51.26.250 - - [03/Dec/2009:23:59:59 -0300] "GET /js/zoom.js HTTP/1.1" 200 244 "http://www.cpca.org.ar/contenido/a835_Sorteo%3A-Navidad-en-Comunidad" "Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15"
190.51.26.250 - - [03/Dec/2009:23:59:59 -0300] "GET /js/zoom.js HTTP/1.1" 200 244 "http://www.cpca.org.ar/contenido/a835_Sorteo%3A-Navidad-en-Comunidad" "Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.15) Gecko/2009101601 Firefox/3.0.15"
201.231.55.139 - - [03/Dec/2009:23:59:59 -0300] "GET /images/contenidos/blt_articulos_mas_populares.gif HTTP/1.1" 200 995 "http://www.cpca.org.ar/contenido/s149_Concursos" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
201.231.55.139 - - [03/Dec/2009:23:59:59 -0300] "GET /images/contenidos/blt_articulos_mas_populares.gif HTTP/1.1" 200 995 "http://www.cpca.org.ar/contenido/s149_Concursos" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"


Thank you in advance!

[KingBeto]

Please somebody help!!!