Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 2 > Feature Requests

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 16th September 2007, 13:07
Serg Serg is offline
Junior Member
 
Join Date: Sep 2007
Location: Zaporozhye, Ukraine
Posts: 6
Thanks: 2
Thanked 1 Time in 1 Post
Default Multiplexing webservers with ISPConfig statistics

Hello, everybody!
I'm using ISPConfig some weeks and can say, that is nice software, congradulation to developement team.
But there is a wider client wishes, and I must to solve some problems.
Preface.
There is standard ISPConfig installation.
Also there is port-based hosting due to plone/zope service on rank of ports 2XXX-2X10
Also there is another independend webserver with old feautures like MYSQL 4.0 and PHP4 (XAMPP powered) on port 85.
And I can(and make) multiplexing to all servers via squid proxy.
Main Apache2 now listen port 82.
Control ISPConfig Apache1.3 listen port 81.
Squid listen port 80 and work as transparent proxy according to simple acl rules.
All works fine, but statistics still not.
I reformat squid log outpot as ISPConfig log format.
There is one difference(example)
ISPConfig /var/log/httpd/ispconfig_access_2007_09_10 :
...GET /favicon.ico HTTP/...
Squid(I point squid log to same place and make needed symlink to it) :
...GET http://test.domain.net/favicon.ico HTTP/...
Then I point main apache2 log file to /dev/null
Squid log utilization is made by my script in cron at 00.01 am, it unlink ispconfig_access_log from old log, delete two-days-old log, creates new day log, put symlink to it and then reconfigure(force reload) squid.
But at morning I see two separate directory in /var/www with same name.(test.mydomain.com was created by ISPConfig panel)
One is test.mydomain.net - symlink, as known to /var/www/webuserID with old log, generating when apache2 stored log directly and belonging to virtual host test.mydomain.net
Second is real direcrory test.mydomain.net with new(not empty,right!) log, belonging to test site and generating from squid log.
First directory is owned by testclient:webuserID
Second is owned by root:root
Directory creation time is 00:30
Webalizer page operates with the old statistic, but all files in stat/ directory was rewritten in 04:00 am last day.
1.What happen? Why there are two separate dirs with same name(but one is symlink)?
And how I can tell script to write site log to correct dir/file(eq symlink)
2.Which process is starter of scripts/shell/logs.php ? In 00.30 as I understand?
It seems, it script run automatically by ... program
3. Can I run(for test purpose, of course, not on production host) logs.php script to see changes with logfiles?
testhost#php logs.php (or any parameters required?)
Big thnks for reply

Last edited by Serg; 16th September 2007 at 13:13.
Reply With Quote
Sponsored Links
  #2  
Old 17th September 2007, 16:49
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,748 Times in 2,579 Posts
Default

Quote:
Originally Posted by Serg
2.Which process is starter of scripts/shell/logs.php ? In 00.30 as I understand?
It seems, it script run automatically by ... program
It is run by cron. See
Code:
crontab -l
Quote:
Originally Posted by Serg
3. Can I run(for test purpose, of course, not on production host) logs.php script to see changes with logfiles?
testhost#php logs.php (or any parameters required?)
Big thnks for reply
The correct command is
Code:
/root/ispconfig/php/php /root/ispconfig/scripts/shell/logs.php
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
The Following User Says Thank You to falko For This Useful Post:
Serg (18th September 2007)
  #3  
Old 18th September 2007, 10:04
Serg Serg is offline
Junior Member
 
Join Date: Sep 2007
Location: Zaporozhye, Ukraine
Posts: 6
Thanks: 2
Thanked 1 Time in 1 Post
Default

Thank you, Falko

Quote:
Originally Posted by falko
The correct command is
Code:
/root/ispconfig/php/php /root/ispconfig/scripts/shell/logs.php
Yes! When I run this script with the analyze some variables (thanks to print_r() ) I catch an error - finishing SPACE in filenames. This was my (hidden ) bug in Logformat string. Shame to me!
And now it fixed and all OK.
Is squid-with-ISPconfig integration guide necessary for anyone?

Great! We have now nice statistics of port-based hosting and (even!) of co-location traffic to another hosts with their web(ftp, etc)-servers. All clients can to operate statistics with ISPConfig client's web face and we can make new hosting plans and invoices.
In this case there is a idea - to prepare a "tiny" client's interface or make a possibility to construct client ifaces by admin, depend of client's needs (eq with/without WEB-FTP, with/without database/co-domain/mail/user management, less menus items and so on)
Any suggestions?
Reply With Quote
  #4  
Old 18th September 2007, 11:17
the_spy the_spy is offline
Member
 
Join Date: May 2007
Posts: 91
Thanks: 20
Thanked 6 Times in 6 Posts
Default

Quote:
Originally Posted by Serg
Is squid-with-ISPconfig integration guide necessary for anyone?
can you please explain how is working your installation of squid ? I don't understand how it works in squid and which configuration is needed to distribute requests to one port service or an other, and it could interest me if you can put a small tutorial here why not with ispconfig statistics also... Thanks
Reply With Quote
  #5  
Old 20th September 2007, 11:26
Serg Serg is offline
Junior Member
 
Join Date: Sep 2007
Location: Zaporozhye, Ukraine
Posts: 6
Thanks: 2
Thanked 1 Time in 1 Post
 
Default

Quote:
Originally Posted by the_spy
can you please explain how is working your installation of squid ? I don't understand how it works in squid and which configuration is needed to distribute requests to one port service or an other, and it could interest me if you can put a small tutorial here why not with ispconfig statistics also.
Well. This is not tutorial, but common issue, because I don't have such time to write full guide now(and in future week too)

We assume, that our apache will listen on port 82, as one of any port-based services and we use Debian
There are steps to prepare hosting service to work in complex:
1. Stop main Apache.
2. Install squid and squidguard by apt-get. (We don't need to use base squid from Debian distro, because it not responces all logformat parameters, discribed in squid tutorial , but we need squidguard)
3. Install squid from source to /usr/local. You can install squidguard from source too and skip step 2 at all.
4. Configure squid. Next code not whole, but touch only engine with ISPConfig
Code:
http_port IP.IP.IP.IP:80 transparent
tcp_outgoing_address IP.IP.IP.IP
emulate_httpd_log	off
logformat squid %{Host}>h||||%st||||%>a %un %ul [%tl] "%rm %ru HTTP/%rv" %Hs %st "%{Referer}>h" "%{User-Agent}>h"
access_log /var/log/httpd/ispconfig_access_log squid

acl all src 0.0.0.0/0.0.0.0
acl to_localhost dst 127.0.0.0/8
acl our_host dst IP.IP.IP.IP

http_access deny to_localhost
http_access allow our_host
http_access deny all

follow_x_forwarded_for allow all
log_access allow our_host
log_access deny all
logfile_rotate 0
url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidGuard.conf
url_rewrite_children 5
url_rewrite_host_header on
# via on
via off #this inject warning in cache log, but works
cache_effective_user proxy
cache_effective_group proxy
digest_generation off
icp_port 0
htcp_port 0
forwarded_for on
You can play with some options such as http_port (transparent, accel, vhost, vport) And we use external redirect program but it is possible use cache_peer directive and point to our apache server as parent cache on port 82 with some successful options, it works too.
5. Configure squidguard
Code:
rew dmz {
	s@://control.youdomain.com@://youdomain.com:81@i
	s@://IP.IP.IP.IP@://sharedip.youdomain.com:82@i
	s@://youdomain.com@://youdomain.com:82@i
	s@://www.youdomain.com@://youdomain.com:82@i
#and something else including all virtual hosts, webmail and redirects to another services
	#Last string. If host exist in DNS but has no service on port. This prevent loop
	s@://.*\.youdomain.com@://no-in-serve.youdomain.com:82@i
}
dest local {
}
acl {
	default {
		pass	 local any
		rewrite	 dmz
	}
}
6. chown -R /var/log/httpd to rootroxy
chmod /var/log/httpd to 775
make a file /var/log/httpd/ispconfig_access_log_2007_M_D
where M and D - current month and day
make a symlink to it with ispconfig_access_log
7. start and testing squid.
8. Patch /root(orwhereisyouinstallISP)/ispconfig/scripts/lib/config.lib.php
Near string No 1264 write port :82 instead :80
9. Patch apache's conf
ports.conf
Listen:82

apache2.conf
Comment (#)LogFormat and CustomLog
Put new directives:
SetEnv dontlog 1
LogFormat "%v %b" combined
CustomLog /var/log/something../access.log combined env=!dontlog
10. Start apache. Go to URL youdomain.com:81 (to ISPConfig) and create new hosts with DNS records: control.youdomain.com shareip.youdomain.com and no-in-serve.youdomain.com, put to web directory on last two hosts nice pages with text.
All you hosts in Vhost.conf will updated to port 82 automatilally.
11. Now you can point you browser to control.youdomain.com and see login page of ISPConfig.

Thats not at all. You must take care to rotate logs and to remake correct symlink and permissions in log/httpd, after logs rotation you must reload squid. This is may be done by our frend Cron Oliveira.
Also we must update check_services.php about 82 port.

Now all works, but there are some manual operation when you create or delete site - you must put(or delete) record in squidguard.conf
Work to make this automatically is in progress.
Also squid make different records in logfile about host which has aliases
Another words, requests to domain.com and www.domain.com has separate records. This problem is half-solved(alpha version ) and I not describe jet.
Reply With Quote
The Following User Says Thank You to Serg For This Useful Post:
falko (21st September 2007)
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
fastcgi and php with ispconfig tosser Installation/Configuration 14 18th January 2009 14:41
ispconfig together with other portal app Mr_Miyagi Installation/Configuration 2 24th March 2007 18:19
fastcgi and php with ispconfig tosser Tips/Tricks/Mods 3 25th June 2006 22:01
Upgrading failed, ispconfig broken and ispconfig won't upgrade again dfriis Installation/Configuration 7 14th June 2006 18:54
ISP-Ubuntu breezy ispconfig dns email e-mail working finally yellowjelly HOWTO-Related Questions 0 19th November 2005 21:30


All times are GMT +2. The time now is 23:59.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.