Replacing the dead master server in ISPconfig multiserver mirror configuration

Discussion in 'ISPConfig 3 Priority Support' started by gscaglia, Jul 13, 2018.

  1. gscaglia

    gscaglia Member HowtoForge Supporter

    Hi Till,
    I have in production two identical servers with Debian 9, in different datacenters, configured in mirror with ISPconfig 3.1 multiserver for email and web services.

    Suddenly the master server for ISPconfig has died and I can not recover data in short time.

    So as not to leave the slave server alone, I decided to install a new server from the beginning and copy the data from the current slave in order to restore the mirror as soon as possible. But how to do it? Which files of Apache, Postfix and PureFTPd should I copy? And for users and passwords (mailboxes, ftp) is it enough to copy the ispconfig database of the slave into the new master's one or must I copy some Debian files too? Is there an how-to to solve this situation?

    Certainly I will have to copy the folders and files of /var/vmail/ - /var/www/ - /var/lib/mysql/ - /etc/bind/ - /etc/letsencrypt/ archive/ - /etc/letsencrypt/live/ and ISPconfig database from the slave to the master (after the new ISPconfig master installation).

    Is there an easier way to do all this?

    Thanks a lot
  2. Taleman

    Taleman Well-Known Member HowtoForge Supporter

  3. gscaglia

    gscaglia Member HowtoForge Supporter

    In my case the mysql master/master replication was interrupted about a month ago in one direction (ISPconfig master server -> ISPconfig slave server: stopped; the mail and webservices were public on the slave server, as now) and so now, on the server slave, I have both databases (dbispconfig and dbispconfig2) but noone is correct.
    Also because I am continuing to make changes on the server slave in these days and I'm using the ISPconfig panel interface on the server slave. I see that ISPconfig interface, on the slave, write the change only on dbispconfig database: so I'm using the ISPconfig on slave server, the survivor, but the database updated is dbispconfig (in teory, the database of master server). :confused:
    Can this be a problem for rebuilding the ISPconfig new sever master (and for the subsequent resynchronization of the slave)?

    Could I use the ISPConfig Migration Toolkit to move everything from the slave server to the new master server?
    Last edited: Jul 14, 2018
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    I guess you have to decide which database you use, either dbispconfig or dbispconfig2, use the one which has the better data. If you can join data, probably not, because ID's in the different tables are probably used by diifferent data in the two databases now (almost every table has a primary ID column, that's the one i refer to, and all the data (e.g ftp users to websites is interconnected based on these ID's.)

    The Migration tool will not work, but ISPCopy, which is part of the Migration toolkit as well, might work as it copies and ispconfig installation 'as it is' by ssh from one server to another one (it copies passwd, shadow and group file records, the ispconfig database and wen databases, websites, emails, config files from used services like postfix, dovecot, pure-ftpd, bind and the ispconfig code itself /usr/local/ispconfig). But I must admit that I have not used it in this special situation yet, so there is no guarantee.
    ahrasis likes this.
  5. gscaglia

    gscaglia Member HowtoForge Supporter

    Thanks Till.

    I should understand better how ISPconfig multiserver works to decide what to do, because I noticed that from the web interface (of ISPconfig) installed on the slave server (of ISPconfig) I can not manage the slave server itself (ie the survivor). The panel works, but the changes are not writed on the services configuration files on the disk (and are written to dbispconfig database, instead of dbspconfig2 as I would expect), as if the slave could not act in the absence of the ISPConfig master installation up.

    So I ask you:

    1) Is it normal that the ISPconfig panel, actived on the slave, can not manage the services, if the server where ISPconfig's master is installed is down, or there is something wrong in my configuration?

    2) To fix the situation quickly I could e.g. transform the slave installation of ISPconfig into a master, so I can operate on the surviving server?

    3) You can explain me what happen between the two ISPconfig installations (master and slave) and which databases they writing, both when inserting a modification on the ISPconfig master panel, and when inserting a modification on the ISPconfig slave panel (the red sticker with the number of the changes, in ISPconfig, disappears because they are compared the two databases, dbispconfig and dbispconfig 2, both present on it mysql server)?

    Thanks a lot
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    Yes, that's normal.

    What you can try is that you change the name of the database that is used by ispconfig on the second server in the file /usr/local/ispconfig/server/lib/ from dbispconfig2 to dbispconfig.

    An ispconfig multiserver system has just one master database, no matter if you use a mirror or not. This database is normally named dbispconfig and the ispconfig interface connects to it. Each slave server has it's own database, in your case 'dbsipconfig2'. The slave server connects to the master database on the first server, pulls the changed data and inserts it into its local database.

    So how does this work on the mirror now that you can have two interfaces? The trick that is sued here is a combination of the ISPConfig master / slave data transfer mechanism with mysql master/master replication. In a mirror setup, the master database dbispconfig exists on both servers and has the same content due to mysql master/master replication in place, same with the dbispconfig2 database. So how do the servers connect to the databases:

    Interface connects to dbispconfig on localhost
    Server connects to dbispconfig on localhost

    Interface connects to dbispconfig on localhost
    Server connects to dbispconfig2 on localhost

    The server part of server2 now connects to dbispconfig on the master and pulls data into dbispconfig2 once a minute.
    ahrasis likes this.
  7. gscaglia

    gscaglia Member HowtoForge Supporter

    Thank you so much Till, now I know something more about how the multiserver works.

    The good database was 'dbispconfig2' so I made it become 'dbispconfig'; I also made some changes to some tables and database permissions.
    I changed the database name settings in the file /usr/local/ispconfig/server/lib/ like you said, however, even if the panel seems to work fine, no changes are made to the disk and never appears, even for a moment, the red dot.
    It seems that the server part of ISPconfig continues to work on dbispconfig2, any ideas?

    About the red dot: in a functioning multiserver configuration, both the server part of the ISPconfig master and the server part of the ISPconfig slave, write (eg a flag) in the dpspconfig database on the master, to confirm to have performed the operations on disk, is it correct?

    To avoid these inconveniences in the future, I'm thinking of changing the installation architecture of ISPconfig in this way. In a VM, on the physical machine A, I only will put the master version of ISPconfig and nothing else. On two VMs in two different physical machines (A and B) all services (DNS, emails and web sites) with a slave ISPconfig installation. Finally, in a VM on the physical machine B, a sleeping ISPconfig master installation that remains aligned, to the ISPconfig master in use, with the mysql replica of dbispconfig.
    If the physical machine A should fall, it would be enough to change the IP address of eg. in the /etc/hosts file of the VM with the services on B and take it controls from the ISPconfig master on the VM on B. In this way I should be able to use ISPConfig 3 Migration Toolkit without problems if I'll have to migrate the services to one new machine.
    Do you think it's a good solution?

    Thanks for your patience
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    Actually I missed one part :) Please edit the file /usr/local/ispconfig/server/lib/ file (make a backup of it first) and empty the variables for the dbmaster connection. Currently, the system still tries to connect to the master to fetch changes. And one thing that I'm not 100% sure is if you might have to alter the server_id from 2 to one in all config files on the server (for simplicity, edit only the two files from ispconfig and then do an ispconfig update with reconfigures services to let the ispconfig updater do the other files). Try the dbmaster change first, if it still does not work, then proceed with the server_id change.

    I think this should wor. ISPConfig is quite robust in this regard. The interface(s) must write to the fisdrt (master) database and the slaves must have their own database and the dbmaster details of the slaves mut point to the master sql database, that's basically the whole requirement for a working setup, so there is plenty of room how to actually implement it, the two multiserver setups that we publish are just two possible solutions, but there are many more possibilities.

Share This Page