Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 2 > Developers' Forum

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 30th November 2005, 10:33
kea13 kea13 is offline
Junior Member
 
Join Date: Nov 2005
Posts: 7
Thanks: 0
Thanked 0 Times in 0 Posts
Default OT? a (possible,future) bigger picture

Hello everybody,

i've been hesitating to post this for a couple of days, but today is a good day to become flamebait of the week - so here goes... (if this is off topic, not of interest, utter crap, ... - feel free to moderate it into oblivion)

Since a comparison between different control panels and ISPConfig was asked for in a different thread and ISPConfig v3 is going to include support for managing multiple servers, i'd like to propose something ..um.. pretty radical as a possible roadmap for the future

Please have a look at the attached diagram.

Still with me? Good, then let's have some explanations.
With a single machine control panel you have a MTA,Spam-Checker,Antivirus,Control Panel,Webserver,Databaseserver,... on every machine you deploy. Why not distribute the services onto different machines? Do it the H-Sphere or Helm way: start small with a dedicated control panel server and a web/db server and dedicate one or more machines per service as you grow.
That way you have a single point of contact for the users to sign up and administer their hosting package(s). With a dedicated configuration server you can even have more than one control panel servers.
Need more horsepower for the webservers - pop in another box and distribute the customers. If set up as a SSI-cluster this should be close to a no-brainer - talk about power on demand (TM). Same goes for the Database boxes.

In the following I'd like to add a couple of remarks concerning "Logfile Analysis", "Monitoring and Surveillance", "Config Server" and "Content Inspection":
"Logfile Analysis":
leave the processing power where it belongs: to the customers webpages. Creating statistics for 50 or 100 customers may not be much of an impact, but what if you have a "couple" more on a box ? Additionally, traditionally every machine would do reverse DNS lookups for the stats - why not centralize the DNS-cache one one box and add some spiffy things like GeoIP to it ?

"Monitoring and Surveillance":
primarily of interest in order to get a heads up as soon as a parameter shows a tendency of leaving the green sector. As a side effect, this can be used to give the customers some insight into the performance of their hosting package and to prove that the service was inside the SLA.

"Config Server":
in the depicted setup, this service clearly is the achilles heel. You fry this box , you're toasted. Given the easy replication of LDAP this situation might be ameliorated a bit. As a side note one might replicate the list of known users to the SMTP servers in order to reject messages without valid recipient before they enter your system (thereby avoiding Spam- or Virus-bounces).


"Content Inspection":
This service does Spam and Virusscanning for inbound, outbound and "deflected" (more to that in a second, hold on) messages on a per domain or even per user basis. A single point of inspection (service-wise not box-wise
allows for the deployment of a commercial antivirus-solution with much less financial impact than the integrated one-box-does-it-all approach. Not that clamav wasn't a good thing - i just think that certain commercial scanners are doing an exceptional job concerning detection rate and fast updates.
If you're doing business in Germany you'll sooner or later have to cope with a very strange law forcing you to make provisions to hand over all inbound and outbound email along with login/logoff timestamps for a given customer to federal agencies (and I'm sure other countries already have or will have legislation like this in the near future) . This single point of inspection should be able to support just that.
If you want emails that one of your customers sends to another one of your customers (Customer to Customer messages) to be subject to scanning and potential "federal access", the emails will have to go through Content inspection and be handed ("deflected") back to the POP/IMAP servers.

Still with me? Wow, I'm impressed - so whaddaya think about this? Provided that there exists a growth path in the like of H-Sphere and Helm, could this be a road for future development?
So, Ladies and Gentlemen, please start your flamethrowers (but grant me a head start to scramble for shelter

Thanks for bearing with me,
Roman
Attached Files
File Type: gz ISPConfig_vX.jpg.gz (80.5 KB, 472 views)
Reply With Quote
Sponsored Links
  #2  
Old 30th November 2005, 11:38
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,741 Times in 2,575 Posts
Default

I post your image here again so that everyone can have a look...
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #3  
Old 4th December 2005, 13:47
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 36,046
Thanks: 826
Thanked 5,386 Times in 4,231 Posts
Default

Hi Roman, it is planned that ISPConfig 3 will support multiple servers. This means that services can be all on one server or split to many servers, e.g. 1 Server for the controlpanel, 3 Webservers, 1 database server, 2 mailservers, etc.

The configuration information will be stored inside a mySQL database. This database can be replicated to the other servers for faster access with the mySQL replication or with a builtin replication in ISPConfig. Later we might support also postgresql or LDAP.

Currently the splitting is planned between these services:

1) Controlpanel
2) Webserver
3) Mailserver
4) Database server
5) DNS Server

When I read the current news in germany, a central TkÜV Server might be nescessary too (

Maybe we can split it even more fine grained.

Till
Reply With Quote
  #4  
Old 4th December 2005, 14:27
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,741 Times in 2,575 Posts
Default

Quote:
Originally Posted by till
When I read the current news in germany, a central TkÜV Server might be nescessary too (
1984 is coming finally...
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #5  
Old 8th December 2005, 15:36
kea13 kea13 is offline
Junior Member
 
Join Date: Nov 2005
Posts: 7
Thanks: 0
Thanked 0 Times in 0 Posts
Default

Hi Till,

that's great news! I thought that v3 would support multiple servers in the sense of one control panel managing several multi-purpose servers, each with their own SMTP,HTTP,POP3,MySQL/PostgreSQL servers.
With regard to our "beloved" TkÜV: If there was an option (something like "enable TkÜV-compliance" you could maybe force the MX for each of the hosted domains to hit an appropriately configured server.
That way you'd have two views upon your SMTP subsystem: the external one which gives tkuevmx1 and tkuevmx2 (as an example) as MX-records for a domain. On these two mailservers you'd have routing information specifiying onto which of the ISPConfig-managed mailservers the message would have to go - actually a pretty straightforward thing with exim
If I remember correctly it was widely expected that the number of customers which requires TkÜV compliant operations would be raised from 1.000 to 10.000 during one of the last sessions in parliament. The fact that this did not happen should probably be seen as a clear indication as of which way we're heading...

And now for something completely different:
Have you guys thought about SSI-clusters for the services?
Reply With Quote
  #6  
Old 8th December 2005, 16:27
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,741 Times in 2,575 Posts
Default

Quote:
Originally Posted by kea13
Have you guys thought about SSI-clusters for the services?
What are SSI-Clusters?
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #7  
Old 8th December 2005, 17:00
kea13 kea13 is offline
Junior Member
 
Join Date: Nov 2005
Posts: 7
Thanks: 0
Thanked 0 Times in 0 Posts
Default

Single System Image Clusters:
Short Version: Single Root Filesystem, i.e. single copy of binaries and configuration files. Need more availability: boot another box into the cluster. Need more performance: boot another box into the cluster. Same goes for the other way round.
Basically it's a cluster with a single filesystem which lets you add/remove machines as needed. Pretty slick especially considering the growth path: start with a moderate machine now. If you tend to outgrow it there's no need for downtime to upgrade the hardware, simply add another box (or more).

For the long story you might want to visit http://www.openssi.org
Reply With Quote
  #8  
Old 8th December 2005, 21:18
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 36,046
Thanks: 826
Thanked 5,386 Times in 4,231 Posts
Default

SSI clusters are looking really interesting. If i understand it right, you can simply replace a server managed by ISPCOnfig by an SSI cluster. If the cluster behaves like one server, there is no special configuartion nescessary to use it with ISPConfig?
Reply With Quote
  #9  
Old 9th December 2005, 00:17
kea13 kea13 is offline
Junior Member
 
Join Date: Nov 2005
Posts: 7
Thanks: 0
Thanked 0 Times in 0 Posts
Default

i'm pretty sure that's exactly the way it's going to be - haven't tested it yet but will gladly do so (hopefully in the near future)
In case i get around to do it before you guys - anything special i should look for/test?
Reply With Quote
  #10  
Old 2nd February 2006, 05:41
webstergd webstergd is offline
Member
 
Join Date: Dec 2005
Location: Washington, DC
Posts: 53
Thanks: 0
Thanked 0 Times in 0 Posts
 
Default addition

It would be nice to add snort to this. Shouldn't be hard to make custom rulesets that would ignore our information being passed.

we would need to work with the log files...If the systems starts to gain traffic snorts log files get huge. You can limit this with a basic config ohh so much.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 06:33.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.