ISPConfig multiserver setup with pacemaker and DRBD for syncing/switching with backup server?

Discussion in 'ISPConfig 3 Priority Support' started by Tomislav Aurednik, Jan 18, 2017.

  1. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Is this "ISPConfig multiserver setup with pacemaker and DRBD for syncing/switching with backup server" even possible?
    I want to have 2 servers (server1 and server2) that are running KVM hypervision, each with 4 virtual servers for web&mysql, mail, dns1 and dns2. Every virtual server will have drbd and pacemaker installed to replicate to the node on the other server (server1.web&mysql <-> server2.web&mysql2).
    I know that DRBD needs to have it's own partition that it replicates. So how to install to that partition and what do I have to replicate?
    How to configure ISPCinfig on the second server. Do I set all virtual servers on server2 to mirrors to be mirrors of the virtual servers on server1?
    Or do I have to install a new multiserver setup on the server2 and have it's ovn primary server with the interface?
    Any other thing I shold be aware of?
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    So each of the vm has one mirrored counterpart, right? Do you do the drbd replication on the vm level? If yes, then there is no setup in ISPConfig required as ispconfig is not aware and does not has to be aware that you replicate it.

    If you don't do a replication on vm level, then you can find a guide to setup an ispconfig mirror in the ispconfig manual.
     
    Tomislav Aurednik likes this.
  3. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Thanks for the reply!
    I wan to do the replication on the vm level. Every vm will have a drbd partition and that will replicate to the coresponding node. But for every server I have to put the files that will replicate to that partition. I'm in contact with Linbit and they told me that for the DB I need only store the DB files on the drbd partition and replicate it. But won't that overrite the needed dbisconfig table on the node server?
    Can I have the ispconfig interface on the primary server and on his node, that when pacemaker switches the server, it has a usable interface?
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    If you replicate on the vm level, then the software inside the vm is not aware of the replication and it has not to be aware of that as the hypervisor takes care on replicating everything. The answer that you posted from linbit is not about a vm level replication, so there might be some misunderstanding here. When they speak about different services like mysql, then the vm is not replicated, instead you just have a shared hard drive that is shared between several servers like an NFS drive.

    So if you just have a shared network drive within your vm, then you have to replicate all services as described in the mirror tutorial of the ispconfig manual. If you look at that tutorial, you can see that the ispconfig databases on both nodes have different names, dbispconfig1 on the first server and dbispconfig2 on the second server, so they do not get overwritten. So in your setup you replace the mysql replication with a drbd setup. Then you will have to mount /var/www onto the drbd drive so that the website files are shared between the servers if you don't want to use unison for that.
     
  5. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Thanks again. No there is no shared NFS drive. Both vm have drbd and pacemaker installed, and each server a drbd partition that is replicated. The shared hard drive is no option while using LVM partitons because they'll overrite the data.
    Didn't think of using different names for the db tabel.... thanks for this.

    I'll try to use a combination of this tutorials with your howto for multiserver setup.
    https://www.globo.tech/learning-center/setup-drbd-9-ubuntu-16/
    http://stackoverflow.com/questions/1795176/how-to-change-mysql-data-directory
    http://wiki.clusterlabs.org/wiki/DRBD_MySQL_HowTo

    But I just needed to know that it's possible and thank you for that! But if you have any pointers that I have to be aware, they would be appreciated!
     
  6. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    I would not use drbd for database and mail-replication if you need just two servers for each service. I would use mysql master-master and dovecot's dsync. You can find my tutorial in the ispconfig manual.
    I stopped using drbd a few years ago and switched to rsync for websites. But maybe drbd is working better nowadays.
     
  7. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Drbd works quite well for mysql and web replication. I tryed rsync it's not bad but it's more for a backup server than a HA server. At leats I used it only one-way replication. Drbd is two-way. It's even quite simple to create the resources, but the pacemaker crms are a bit trickier.
    A single server install is quite easy.
    You create drbd resources for apache (the sites, mods, avalible and enabled), drbd resource for wesites (/var/www), drbd resource for mysql (/var/lib/mysql) and mail (/var/vmail). Maybe even one for certs. Then you install until the ispconfig part, mount the resources to tmp dirs and move the "correct" files ontu them. After that you mount them on the "correct" locations.
    Than you need to create the pacemaker crms for auto mounting the resources, starting the services, master-slave selection and floating IP.
    For the ispconfig install you then just install it on the primary node, poweroff the node so the 2nd node becomes primary. Delete the ispconfig database and run the install again (I used the same database user password for both installs... the auto generated one ).

    But the my problem now is that I can't figure out how to install a multiserver install. Exactly I can't figure out how to join an existing ISPConfig multiserver setup during install?

    An out off place question/s: Do you have any comercial/presentation (screenshots or videos) for your billing module and sever migration tool?
    I'll be needing them after I figure out how install the ha multiserver setup and install it.
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    During ISPConfig installation, select "Expert" install mode, the installer will ask you then if this node shall join a master server.

    No. All available information can be found on ispconfig.org website, for the billing module, there is a online demo as well, you can find the link on the billing page at ispconfig.org.
     
  9. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Till I know that and I do that. The problem is after that. During the install when it trys to connect to the master server. I created the db users in the DB off the master server for the floating IP and floating IPs hostname (didn't even connect), created the users for the active nodes IP and hostname. That time the install went foward but crashed with fatal php error.
    PHP Fatal error: Uncaught Error: Call to undefined function mb_detect_encoding() in /tmp/ispconfig3_install/install
    /lib/mysql.lib.php:445
    Stack trace:
    #0 /tmp/ispconfig3_install/install/lib/mysql.lib.php(146): db->escape('dbispconfig')
    #1 /tmp/ispconfig3_install/install/lib/mysql.lib.php(228): db->_build_query_string('CREATE DATABASE...', 'dbispconfi
    g', 'utf8')
    #2 /tmp/ispconfig3_install/install/lib/mysql.lib.php(267): db->_query('CREATE DATABASE...', 'dbispconfig', 'utf8')
    #3 /tmp/ispconfig3_install/install/lib/installer_base.lib.php(237): db->query('CREATE DATABASE...', 'dbispconfig', '
    utf8')
    #4 /tmp/ispconfig3_install/install/install.php(290): installer_base->configure_database()
    #5 {main}
    thrown in /tmp/ispconfig3_install/install/lib/mysql.lib.php on line 445
    How should I try to connect the multiserver setups with the floating IPs/hostnames or the node IPs/hostnames?
    my hosts on master server nodes (I won't paste both, they're the same except for the IP and hostname for the 1st and 2nd node are swiched)
    127.0.0.1 localhost
    192.168.123.188 test1.mydomain.si test1
    #2nd node
    192.168.123.189 test2.mydomain.si test2
    #Floating IP mysql server
    192.168.123.197 testmysql.mydomain.si testmysql
    #Floating IP web server
    192.168.123.187 testweb.mydomain.si testweb
    # The following lines are desirable for IPv6 capable hosts
    ::1 localhost ip6-localhost ip6-loopback
    fe00::0 ip6-localnet
    ff00::0 ip6-mcastprefix
    ff02::1 ip6-allnodes
    ff02::2 ip6-allrouters
    ff02::3 ip6-allhosts​
    my interfaces on master server nodes (not same on both nodes... the required diff. and I hide some IPs)
    auto ens3
    iface ens3 inet static
    address 192.168.123.188
    netmask 255.255.255.0
    network 192.268.123.0
    broadcast 192.168.123.255
    gateway 192.168.123.x
    dns-nameservers x.x.x.x x.x.x.x
    auto ens3:1
    iface ens3:1 inet static
    address 192.168.123.187
    netmask 255.255.255.0
    On the mysql nodes that I want to join.
    127.0.0.1 localhost
    192.168.123.198 tmysql1.mydomain.si tmysql1
    #2nd node
    192.168.123.199 tmysql2.mydomain.si tmysql2
    #Floating IP mysql server
    192.168.123.197 testmysql.mydomain.si testmysql
    #Floating IP web server
    192.168.123.187 testweb.mydomain.si testweb
    # The following lines are desirable for IPv6 capable hosts
    ::1 localhost ip6-localhost ip6-loopback
    fe00::0 ip6-localnet
    ff00::0 ip6-mcastprefix
    ff02::1 ip6-allnodes
    ff02::2 ip6-allrouters
    ff02::3 ip6-allhosts

    iface ens3 inet static
    address 192.168.123.198
    netmask 255.255.255.0
    network 192.168.123.0
    broadcast 192.168.123.255
    gateway 192.168.123.x
    dns-nameservers x.x.x.x x.x.x.x

    auto ens3:1
    iface ens3:1 inet static
    address 192.168.123.197
    netmask 255.255.255.0​
     
  10. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Oh thanks... I'm looking at the demo! tottaly forgot about ispconfig.org site :D For creating the pdf invoice can I use a html template? I created a web aplication for creating invoices from xml and I have my template created for mpdf to create it.
     
    Last edited: Apr 4, 2017
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    The php mbstring module is missing on your server.
     
    Tomislav Aurednik likes this.
  12. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Hm... I only installed this modules on my mysql server:
    apt-get -y install php7.0-cli php7.0-mysql php7.0-mcrypt mcrypt
    used the multiserver setup for referance.

    Why do I need to add sql-mode="NO_ENGINE_SUBSTITUTION" to my mysql config before I can install ispconfig? haven't seen this option in any howto?
     
  13. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    ok this worked. I could install the servers.
    Installed on the 1st node, poweroff it so the master became 2nd node. Deleted the ispconfig database and the server record in ispconfig cp for the mysql server and installed again.
    Now the problem is that it does not finish the job quene when I try to create db user & db. There are no debug/warnning/error messages in system-log. the permisions on var/lib/mysql are mysql:mysql. During the install I added the 3306 port for the MySQL master server. Should I left that empty?
     
  14. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    the slave servers can't connect to the master setup while running the /usr/local/ispconfig/server/server.sh script.
    I get error
    PHP Warning: mysqli_connect(): (HY000/1130): Host '192.168.123.198' is not allowed to connect to
    this MySQL server in /usr/local/ispconfig/server/lib/classes/db_mysql.inc.php on line 79
    If I add a root user with the ip I get error:
    PHP Warning: mysqli_connect(): (HY000/1045): Access denied for user 'ispcsrv2'@'192.168.123.198'
    (using password: YES) in /usr/local/ispconfig/server/lib/classes/db_mysql.inc.php on line 79
    have only this user for floating IP 192.168.123.197 and after I add him and give him all priviliges it still won't connect.
    So it can't populate changes to server :/
     
  15. till

    till Super Moderator Staff Member ISPConfig Developer

    These are all hostname setup/inconsistency issues of your systems. If the IP's and hostnames are inconsistent, then the setup will fail. Check the /etc/hosts files on all nodes and ensure that the hostnames used for the connections are defined there and that you added a root user which is allowed to conect from these host names and IP addresses that are defined in all /etc/hosts files for all nodes are able to connect to the master.
     
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    MySQL connects are never made to a floating IP, MySQL connects always to a fixed IP. Most likely you accidently used a floating IP in one of the hosts files or you used a hostname for a node that resolved to a floating IP instead of the static IP of the node.
     
    Tomislav Aurednik likes this.
  17. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    yes I used the hostname of the floating IP because the floating IP points allways to the Primary node that has the the file system loaded, started all the services and the public IP points to it.
    Else if I had both nodes IP, it would be like having 2 different servers and clients could select the server for creating the site on the wrong node that does not have the filesystem mounted for example in /var/www.

    So such a setup is not possible?
    [​IMG]
     
    Last edited: Apr 4, 2017
  18. till

    till Super Moderator Staff Member ISPConfig Developer

    The problem with your setup is that ispconfig is normally doing the config mirroring and in your setup, it is unknown if a change reached the right server, so ispconfig can not do that anymore for you. What you are basically doing is that you try to setup a mirror outside of ispconfig, so ispconfig is not involved in configuration of the cluster at all anymore and it can't do that of course at it does not even knows the servers due to your floating IP's. You can probably do such a thing as active / passive cluster so that you just install the three (1) servers and then use a tool like rsync to keep their passive counterparts (2) in sync, but you won't install ispconfig on the second servers then as the complete setup is synced from the first server anyway.
     
  19. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Do you mean sync the whole disk? Wouldn't this bring complications for hostnames and IPs? I don't understand fully what you meant.
     
  20. till

    till Super Moderator Staff Member ISPConfig Developer

    The problemm is this: a normal ispconfig mirror setup uses a shared disk which is accessible from both nodes. You want to use a disk that is mounted on one node only, but without the /var/www disk, ispconfig will not work on the second node as it is not able to do the mirrored actions like creating a new website etc. without the disk. So you can not setup an ispconfig mirror like this, so what remains when you want to have the disk mounted at one node only is that you have to find a way to setup ispconfig on two servers without ispconfig mirroring in a way that the second ispconfig instance is a copy of the first one but inactive, as it may not run its own cronjobs. Not shure if this is easy to achieved as I never tried it.
     
    Tomislav Aurednik and Samo_slo like this.

Share This Page