Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 2 > Installation/Configuration

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #11  
Old 30th November 2005, 14:26
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 35,981
Thanks: 825
Thanked 5,371 Times in 4,218 Posts
Default

Quote:
Originally Posted by RotHorseKid
The problem is, AFAIK ISPconfig is using the Apache logs to find out which site is generating how much traffic.
With Squid in front of Apache, Squid serves the content it has cached directly, Apache does not know about that, therefore it will not be in the logs (the log/web_log files found beneath the web directories).
BUT the traffic is generated anyway (at the external interface of my server, where my ISP measures the traffic, they don't care if it's cached or if apache served it), and I (resp. my clients) still have to pay for that.
At least I believe that is how it works. Tell me if I am wrong.
You are about 98% right

Logs and traffic counting for websites in ISPCOnfig:

1) Apache writes one big logfile for all sites. The logfile has the name of the day (cronolog), e.g. /var/log/httpd/ispconfig_access_log_2005_11_30

2) This log is splitted nightly by the script /root/ispconfig/scripts/shell/logs.php. The splitted logfile parts are appended to the logs inside the website directory. The logs.php scripts counts also the traffic while it is plitting the logfile.

3) Later, the webalizer.php script is run to generate the website statistics based on the splitted logfiles for every website.
Reply With Quote
Sponsored Links
  #12  
Old 30th November 2005, 16:17
Ovidiu Ovidiu is offline
Senior Member
 
Join Date: Sep 2005
Posts: 1,262
Thanks: 78
Thanked 24 Times in 20 Posts
Default

Quote:
Originally Posted by RotHorseKid
With Squid in front of Apache, Squid serves the content it has cached directly, Apache does not know about that, therefore it will not be in the logs (the log/web_log files found beneath the web directories).
BUT the traffic is generated anyway (at the external interface of my server, where my ISP measures the traffic, they don't care if it's cached or if apache served it), and I (resp. my clients) still have to pay for that.
At least I believe that is how it works. Tell me if I am wrong.
you're right, I didn't realize that


Quote:
Not exactly.
I serve a web page that has some content coming from a DB. Connecting to this DB is VEEERY expensive, latency-wise (it's Oracle, perhaps you know what I mean, in my case there are round-trips of 1000-1500ms).
What I know is that some of these pages are static over a long period of time (like the items in a webshop for example). So I go ahead and serve these pages with a Cache-Control: public http header. (Think header() in PHP for example).
Squid caches these pages now, and there are no DB connections.
thx for the explanation

now I would be interested in some feedback if you get the logging done so you can allocate traffic to the customers although traffic is giong through squid.
Reply With Quote
  #13  
Old 30th November 2005, 16:25
RotHorseKid RotHorseKid is offline
Junior Member
 
Join Date: Nov 2005
Location: Germany
Posts: 8
Thanks: 0
Thanked 2 Times in 2 Posts
Default

Quote:
Originally Posted by till
1) Apache writes one big logfile for all sites. The logfile has the name of the day (cronolog), e.g. /var/log/httpd/ispconfig_access_log_2005_11_30
So, to get correct traffic reports, I have to make Squid log (in httpd compat mode) to the symbolic link
/var/log/httpd/ispconfig_access_log
and ISPconfig will take care of the rest?

That almost sounds too easy. But ISPconfig is a fscking great piece of software, so it might just work...

Tell me. If it works like this (or almost like this), it would be easy to offer Squid acceleration for ISPconfig as an option...
Reply With Quote
  #14  
Old 30th November 2005, 16:35
Ovidiu Ovidiu is offline
Senior Member
 
Join Date: Sep 2005
Posts: 1,262
Thanks: 78
Thanked 24 Times in 20 Posts
Default

this might be a little off topic but did anyone try the apache2 mod_proxy? http://httpd.apache.org/docs/2.0/mod/mod_cache.html still experimental!
Reply With Quote
  #15  
Old 30th November 2005, 17:04
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,741 Times in 2,575 Posts
Default

Quote:
Originally Posted by RotHorseKid
So, to get correct traffic reports, I have to make Squid log (in httpd compat mode) to the symbolic link
/var/log/httpd/ispconfig_access_log
and ISPconfig will take care of the rest?

That almost sounds too easy. But ISPconfig is a fscking great piece of software, so it might just work...
Almost...
ISPConfig uses a special log format; it is defined in the httpd.conf and looks like this:

Code:
LogFormat "%v||||%b||||%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined_ispconfig
CustomLog "|/root/ispconfig/cronolog --symlink=/var/log/httpd/ispconfig_access_log /var/log/httpd/ispconfig_access_log_%Y_%m_%d" combined_ispconfig
So a log line in /var/log/httpd/ispconfig_access_log would look like this:

Quote:
www.example.com||||2846||||1.2.3.4 - - [30/Nov/2005:17:01:50 +0100] "GET /bla/icons_expanded.png HTTP/1.1" 200 2846 "http://www.example.com/bla" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.12) Gecko/20050919 Firefox/1.0.7"
So you have to configure Squid so that it prepends the web's domain (www.example.com) and the file size to every line.
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #16  
Old 7th December 2005, 18:24
RotHorseKid RotHorseKid is offline
Junior Member
 
Join Date: Nov 2005
Location: Germany
Posts: 8
Thanks: 0
Thanked 2 Times in 2 Posts
 
Default

Thanx for the info. I will report when I got it implemented.

Regards,
RHK
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 18:00.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.