Installing a new setup

Discussion in 'Installation/Configuration' started by Th0m, Sep 30, 2020.

  1. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    Well, my hosting business is growing way bigger than I expected. Installing most things on one server to use my installation as a standalone install was not a good idea after all, so I want to start moving asap to a multiserver setup where I can add servers if needed and if one server is one, it doesn't mean that all clients and all services are down.

    I want to set up my servers as VPS's at a hosting provider that has a limit for traffic. But you can set up a private network to connect your VPS's to each other so they can communicate without any limit. I would then set the local IP addresses in the hosts file. But I haven't setup a cluster like this before, so I was wondering if anyone had experiences with this.

    This would be my scheme:
    upload_2020-9-30_20-6-24.png
    In the future, I could add a third nameserver, second webserver, etc.

    I am also unsure about what to do with my email. I am thinking about adding a highly available load balancer that would distribute the email traffic between 2 imap servers that sync over dsync. When one mailserver would go down, all traffic would automatically go to one of them. I can then easily add a third one later on if we keep growing and the traffic grows.

    I would love to hear your opinions, tips, do's, and dont's :)
     
    ahrasis likes this.
  2. ahrasis

    ahrasis Well-Known Member

  3. till

    till Super Moderator Staff Member ISPConfig Developer

    Your setup looks good to me. What you did, and what I suggest to any user, is to keep the web and DB server on the same system as long as you don't have special requirements like several web nodes of the same (huge) website access one central database. Separating the control panel server is also a good idea to allow your system to scale better. Regarding load balancing for the mail system, I know some setups which do that successfully for many years, one of them is a 6 node cluster with more than 15 thousand mailboxes, so this should work fine and also scales nicely. This large mail system does not use dsync though, they use a commercial storage system, not sure which vendor.
     
    ahrasis likes this.
  4. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    Thanks for your comments. Currently, there is one thing I am not sure about. If I connect all VPS's to that 'local' network, can I use the external adresses in ISPConfig and the local adresses in the hosts file without problems?
     
  5. till

    till Super Moderator Staff Member ISPConfig Developer

    Yes, that should be ok. You might also ask @Croydon, if I remember correctly, he uses a setup with an internal network for the ISPConfig interconnects.
     
  6. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    I updated the drawing:
    upload_2020-10-1_11-47-12.png
    The names between brackets are servers I can add later on.

    Maybe the webmail will connect to the high availability load balancer, if that traffic is not counted in the quota.
    I'm not sure what's the best way to configure the load balancer, will have to figure that out.
     
    ahrasis likes this.
  7. nhybgtvfr

    nhybgtvfr Active Member

    i've done it with aws, if you put everything in one region/availability zone, you can have them all on one subnet and just use those local ip's in the hosts file.
    assign EIP's (public ip's) to each server and use those addresses in the dns zones.
    traffic between vps's uses the internal ip's, so it avoids the extra costs for traffic across EIP's.
    works perfectly fine, and is a standard concept, so it should work with any hosting provider that'll allow you to use private networks.

    what you have in that diagram isn't an high-availability load-balancer though, it's high-availability mail servers.
    for high-availability load balancers, you'd have two load balancers, with a floating ip switching between them, can be done with haproxy. i assume it can be done with amazon ELB's, and also with digitalocean, google, azure cloud services, but i've never tried to set one up with any of those.
     
    ahrasis likes this.
  8. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    My provider calls it HA-IP Pro. I can copy what they tell you, but better read it yourself: https://www.transip.eu/vps/ha-ip/pro/
     
    ahrasis likes this.
  9. nhybgtvfr

    nhybgtvfr Active Member

    ok, looks like they're basically giving you a 'black-box' which is already combining multiple load-balancers and floating ip's and just leaving you to configure the rest of it.
    guess that means it's one less thing you have to learn/worry about. :)
     
  10. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    Exactly ;)
     
  11. ahrasis

    ahrasis Well-Known Member

  12. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    I decided to leave out the HA-IP Pro as adding that and another mailserver will be expensive and the provider currently doesn't support SSL certificates for domains they don't manage for the HA-IP Pro. Hope to start working on this soon.

    I think I'll use the migration tool, not sure yet.
     
    ahrasis likes this.
  13. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    Making progress, now planning how to set up dsync. If anyone has a working setup with 2 mailservers syncing continously over dsync to each other, please let me know :)
     
  14. michelangelo

    michelangelo New Member

    I've a setup running.
    It's actually quite straightforward to setup dsync. Just have a look at the dovecot wiki about it.
    Something else which you may want to think about is how you wanna achieve the switch of the imap/pop3/smtp services to the failover node in this setup. I would recommend to get a failover IP if you are hosting your servers in a datacenter and then put the services that are supposed to be failover proof on the failover IP.
     
  15. florian030

    florian030 ISPConfig Developer ISPConfig Developer

  16. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    The mailserver will be behind a load balancer in the future, as noted earlier. I'll have to wait for my provider to allow certs for domains managed elsewhere.

    Thanks, I will look into that :)
     
  17. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    Making progress... now trying to set up rspamd as neighbours, but keep getting errors. Currently the CORS header seems to be the problem. Anyone that has rspamd neighbours set up?

    Also not sure where to host my third nameserver, I have one in AMS, and one in finland, shall I host my third in London or Canada? Hmmm...
     
    Last edited: Oct 17, 2020
  18. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    Just did, it works. But it isn't really fast in syncing new emails - any idea how often it is synced?

    EDIT: The sync works, but it seems like the notify doesn't so the sync isn't done. Hmm...

    EDIT2: It doesn't really work, it only works when running manually.
     
    Last edited: Oct 20, 2020
  19. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    Fixed, had to add $mail_plugins to mail_plugins = for protocol lda and protocol imap.
     
  20. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    This thread might be renamed to "DSYNC - HELP", lol. I feel like a noob again setting this up.

    Currently have everything working but with one issue: when a email is read on server1, this is not replicated to server2.
    When a email is moved e.g. from Inbox -> Trash, it is replicated, but read / unread isn't. Email stays in the "new" folder on the other host while it is in "cur" on the server that it was read on.

    Does anyone have a idea what the problem could be?
     

Share This Page