Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 2 > General

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 22nd May 2007, 00:44
ISPConfigFan ISPConfigFan is offline
Junior Member
 
Join Date: May 2007
Posts: 6
Thanks: 1
Thanked 1 Time in 1 Post
Question ISPConfig Logfiles (2147483647 bytes)

Good evening,

For an online game ISPConfig is used on 3 servers that host the website. We have recently switched from Plesk to ISPConfig while using a perfect fedora 6 setup how-to. ISPConfig gave us all we need to simply configure a mini-network with basic functions as mail, ftp, vhosts, etc. First of all, thank you (falko & till i believe; wadering around these forums) for developing such a great, and free, tool!

Now let me excuse myself for writing a complete paper about it, but i think forums are the best tool on the internet and just want to give people with the exact same problem a little hand.

One of the servers is used as database, one for serving dynamic content and one for static content. We have over 5 million hits a day now on the webserver and (big guess) even more on the static. The only problem we encouter is the logfile management of ISPConfig, or how it works. I think that because not many people use it for the same purpose as we do i have not been able to find a solution to our problem here nor on other websites.


The Problem
Static content sometimes took long to load. top showed 25.0+ sometimes while the average cpu usage was never higher then 15%.
Also logfiles where filling up the disk (todays logfile reached 2.1 gig at 20:57

We have stopped all logging through apache (done in httpd.conf) on the static server and disabled Webalizer through ISPConfig.
This solved the problem of the disk being full but still 2 logfiles are 'active':
the one called before that reached 2147483647 bytes at 20:57our, which is called ispconfig_access_log_2007_05_21.
The other one is the same file, but from the day before.

Because we were monitoring the website performance we noticed an enormous increase of cpu usage in top around 20:50 this evening... At the same time the logfile 'stopped' because the time of last change did not increase.
Looking at the dir list i noticed the two logfiles being exact the same size!

Code:
-rw-r--r-- 1 root root 2147483647 May 20 18:17 ispconfig_access_log_2007_05_20
-rw-r--r-- 1 root root 2147483647 May 21 20:57 ispconfig_access_log_2007_05_21
Next thing on my mind is: "the number of bytes of that file is too big, something can't handle it"...
same thing happens if you try to put a larger integer in a mysql field with the 'int' as type: it will be lowered to 2.1 billion.
also, logfiles bigger than 2gigabytes will not be handled by PHP, but this causing the problem didn't make any sense in my mind since PHP got nothing to do with it.

So i do

Code:
echo '' > ispconfig_access_log_2007_05_21
and voila: logfile increases again starting from zero, average 'load' in top drops to 0.2 and website is served flawless with maximum visitors online.

Questions to solve the problem
If i'm not mistaking the logfile i'm talking about is for logging the datatraffic for the ISPConfig panel. The files are useless to us anyway because we use the main webserver for basic visitor stats, they clearly waste a lot of disk and cpu performance, we only host 1 website and they now give us a problem as well...

Is there any way to stop the use of the ispconfig_access_log_2007_*_* files in /var/log/httpd/ ?

More interesting for other people is the question, what is causing this problem?

I hope i have made myself clear enough
Looking forward to any reply.

Thanks in advance,

the ISPCofigFan
Reply With Quote
Sponsored Links
  #2  
Old 22nd May 2007, 10:10
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 36,421
Thanks: 834
Thanked 5,499 Times in 4,328 Posts
Default

The logfiles you are talking about are the logfiles of the main webserver and not of the ISPConfig webserver.

The problem is caused by the fact that the php programming language is not able to open files > 2 GB. There is a documented switch to compile php for file support > 2 GB, but we tested it and it does not work on most linux distributions so we can not use this.

If you want to remove the logging, which will disable all web statistics in ISPConfig, remove the web log and webalizer scripts from the root crontab and the log line that logs trough cronolog in your apache configuration file (it is the custom log directive at the end of the file where ISPConfig inserted some lines).
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
The Following User Says Thank You to till For This Useful Post:
ISPConfigFan (22nd May 2007)
  #3  
Old 22nd May 2007, 15:30
ISPConfigFan ISPConfigFan is offline
Junior Member
 
Join Date: May 2007
Posts: 6
Thanks: 1
Thanked 1 Time in 1 Post
Default

Thanks!

Commenting out the ISPConfig line in the apache did the trick and stopped the use of the logfile.

Too bad though that it is PHP again that causes this mess. Daily logfiles of the webserver reach about 1.7gig and i assume they will give the same problem when they reach 2 gig.

We never had these problems on Plesk, with daily log rotation... maybe it's interesting to find out how their logsystem works because i can imagine others who use ISPConfig as well find this error very annoing.

Thanks again for your time.
Reply With Quote
  #4  
Old 29th April 2008, 22:05
phamels phamels is offline
Member
 
Join Date: Aug 2005
Location: Belgium
Posts: 63
Thanks: 4
Thanked 5 Times in 5 Posts
Exclamation

*bump* :-)

Say I also have this problem (Web log files over 2Gb) and I do wish to have web stats (webalizer) for this specific site... How do I solve this ?

- Do I try to compile php with support to handle files > 2GB and risk running into even bigger trouble
- Or ... there's this magic method I haven't heard of yet or overlooked which just popped into your head and wish to share with me thank you very much for that BTW (Too much coffee)

It's a very busy site with ~40.000 unique visits / day and stats would come in handy

Thx in advance...
Reply With Quote
  #5  
Old 29th April 2008, 22:07
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 36,421
Thanks: 834
Thanked 5,499 Times in 4,328 Posts
Default

This is a rather old thread, it is already solved in ISPConfig 2.2.23.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #6  
Old 29th April 2008, 22:16
phamels phamels is offline
Member
 
Join Date: Aug 2005
Location: Belgium
Posts: 63
Thanks: 4
Thanked 5 Times in 5 Posts
Default

Oh ok :-)

It's a relative new server (35 days old).
I am running this site on version 2.2.21, so that explains :-)

Now if I would update to 2.2.23, will that solve my problem without losing any apache log data?


Thx for the very quick reply btw !
Reply With Quote
  #7  
Old 29th April 2008, 22:20
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 36,421
Thanks: 834
Thanked 5,499 Times in 4,328 Posts
Default

Quote:
Now if I would update to 2.2.23, will that solve my problem without losing any apache log data?
As far as I know, the solution is implemented as log rotation, so it might be that you loose logfile data if the file is already too large and the stats have not been processed.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #8  
Old 29th April 2008, 22:37
phamels phamels is offline
Member
 
Join Date: Aug 2005
Location: Belgium
Posts: 63
Thanks: 4
Thanked 5 Times in 5 Posts
 
Default

Well, the file /var/www/webx/log/2008/04/web.log = 2147483647 bytes as the topic states and stopped growing since a few days... So i suppose/hope that it wil nicely split up after the update and that the logging may continue with hopefully some recovery of the /var/log/httpd files that haven't been parsed yet :-)

I will just try to update after making some backup of the logs i allready have at hand.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
ISPconfig after installation cannot reach www.xyz.de:81 Figth4Linux Installation/Configuration 23 6th March 2008 22:38
ISPConfig 2.3.2-dev released till General 9 4th June 2007 10:46
SP-Server Setup - Ubuntu 5.10 "Breezy Badger" - Page 6 (changes) LuisC-SM HOWTO-Related Questions 0 21st April 2006 15:16
Disk Full, temp files in ISPConfig ecorona Installation/Configuration 2 28th March 2006 01:15
Ispconfig and Virtuemart shajazzi Installation/Configuration 7 21st February 2006 20:25


All times are GMT +2. The time now is 05:07.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.