OT? a (possible,future) bigger picture

[kea13]

Hello everybody,

i've been hesitating to post this for a couple of days, but today is a good day to become flamebait of the week - so here goes... (if this is off topic, not of interest, utter crap, ... - feel free to moderate it into oblivion)

Since a comparison between different control panels and ISPConfig was asked for in a different thread and ISPConfig v3 is going to include support for managing multiple servers, i'd like to propose something ..um.. pretty radical as a possible roadmap for the future

Please have a look at the attached diagram.

Still with me? Good, then let's have some explanations.
With a single machine control panel you have a MTA,Spam-Checker,Antivirus,Control Panel,Webserver,Databaseserver,... on every machine you deploy. Why not distribute the services onto different machines? Do it the H-Sphere or Helm way: start small with a dedicated control panel server and a web/db server and dedicate one or more machines per service as you grow.
That way you have a single point of contact for the users to sign up and administer their hosting package(s). With a dedicated configuration server you can even have more than one control panel servers.
Need more horsepower for the webservers - pop in another box and distribute the customers. If set up as a SSI-cluster this should be close to a no-brainer - talk about power on demand (TM). Same goes for the Database boxes.

In the following I'd like to add a couple of remarks concerning "Logfile Analysis", "Monitoring and Surveillance", "Config Server" and "Content Inspection":
"Logfile Analysis":
leave the processing power where it belongs: to the customers webpages. Creating statistics for 50 or 100 customers may not be much of an impact, but what if you have a "couple" more on a box ? Additionally, traditionally every machine would do reverse DNS lookups for the stats - why not centralize the DNS-cache one one box and add some spiffy things like GeoIP to it ?

"Monitoring and Surveillance":
primarily of interest in order to get a heads up as soon as a parameter shows a tendency of leaving the green sector. As a side effect, this can be used to give the customers some insight into the performance of their hosting package and to prove that the service was inside the SLA.

"Config Server":
in the depicted setup, this service clearly is the achilles heel. You fry this box , you're toasted. Given the easy replication of LDAP this situation might be ameliorated a bit. As a side note one might replicate the list of known users to the SMTP servers in order to reject messages without valid recipient before they enter your system (thereby avoiding Spam- or Virus-bounces).


"Content Inspection":
This service does Spam and Virusscanning for inbound, outbound and "deflected" (more to that in a second, hold on) messages on a per domain or even per user basis. A single point of inspection (service-wise not box-wise
allows for the deployment of a commercial antivirus-solution with much less financial impact than the integrated one-box-does-it-all approach. Not that clamav wasn't a good thing - i just think that certain commercial scanners are doing an exceptional job concerning detection rate and fast updates.
If you're doing business in Germany you'll sooner or later have to cope with a very strange law forcing you to make provisions to hand over all inbound and outbound email along with login/logoff timestamps for a given customer to federal agencies (and I'm sure other countries already have or will have legislation like this in the near future) . This single point of inspection should be able to support just that.
If you want emails that one of your customers sends to another one of your customers (Customer to Customer messages) to be subject to scanning and potential "federal access", the emails will have to go through Content inspection and be handed ("deflected") back to the POP/IMAP servers.

Still with me? Wow, I'm impressed - so whaddaya think about this? Provided that there exists a growth path in the like of H-Sphere and Helm, could this be a road for future development?
So, Ladies and Gentlemen, please start your flamethrowers (but grant me a head start to scramble for shelter

Thanks for bearing with me,
Roman

[falko]

I post your image here again so that everyone can have a look...

[till]

Hi Roman, it is planned that ISPConfig 3 will support multiple servers. This means that services can be all on one server or split to many servers, e.g. 1 Server for the controlpanel, 3 Webservers, 1 database server, 2 mailservers, etc.

The configuration information will be stored inside a mySQL database. This database can be replicated to the other servers for faster access with the mySQL replication or with a builtin replication in ISPConfig. Later we might support also postgresql or LDAP.

Currently the splitting is planned between these services:

1) Controlpanel
2) Webserver
3) Mailserver
4) Database server
5) DNS Server

When I read the current news in germany, a central TkÜV Server might be nescessary too (

Maybe we can split it even more fine grained.

Till

[falko]

Originally Posted by till

When I read the current news in germany, a central TkÜV Server might be nescessary too (

1984 is coming finally...

[kea13]

Hi Till,

that's great news! I thought that v3 would support multiple servers in the sense of one control panel managing several multi-purpose servers, each with their own SMTP,HTTP,POP3,MySQL/PostgreSQL servers.
With regard to our "beloved" TkÜV: If there was an option (something like "enable TkÜV-compliance" you could maybe force the MX for each of the hosted domains to hit an appropriately configured server.
That way you'd have two views upon your SMTP subsystem: the external one which gives tkuevmx1 and tkuevmx2 (as an example) as MX-records for a domain. On these two mailservers you'd have routing information specifiying onto which of the ISPConfig-managed mailservers the message would have to go - actually a pretty straightforward thing with exim
If I remember correctly it was widely expected that the number of customers which requires TkÜV compliant operations would be raised from 1.000 to 10.000 during one of the last sessions in parliament. The fact that this did not happen should probably be seen as a clear indication as of which way we're heading...

And now for something completely different:
Have you guys thought about SSI-clusters for the services?

[falko]

Originally Posted by kea13

Have you guys thought about SSI-clusters for the services?

What are SSI-Clusters?

[kea13]

Single System Image Clusters:
Short Version: Single Root Filesystem, i.e. single copy of binaries and configuration files. Need more availability: boot another box into the cluster. Need more performance: boot another box into the cluster. Same goes for the other way round.
Basically it's a cluster with a single filesystem which lets you add/remove machines as needed. Pretty slick especially considering the growth path: start with a moderate machine now. If you tend to outgrow it there's no need for downtime to upgrade the hardware, simply add another box (or more).

For the long story you might want to visit http://www.openssi.org

[till]

SSI clusters are looking really interesting. If i understand it right, you can simply replace a server managed by ISPCOnfig by an SSI cluster. If the cluster behaves like one server, there is no special configuartion nescessary to use it with ISPConfig?

[kea13]

i'm pretty sure that's exactly the way it's going to be - haven't tested it yet but will gladly do so (hopefully in the near future)
In case i get around to do it before you guys - anything special i should look for/test?

[webstergd]

It would be nice to add snort to this. Shouldn't be hard to make custom rulesets that would ignore our information being passed.

we would need to work with the log files...If the systems starts to gain traffic snorts log files get huge. You can limit this with a basic config ohh so much.