can't see backups. no link between folders.

Discussion in 'General' started by nhybgtvfr, Mar 18, 2016.

  1. nhybgtvfr

    nhybgtvfr Member

    Hi,

    i tried testing out the site backup options yesterday. i set one site to backup daily, and keep 10 copies.

    checking it this morning, it doesn't show anything under existing backups.
    if i ssh to the server, and look in /var/backup, there is a folder there named web22, owned by web22:client1
    inside that folder is a file web22_2016_03_18_00_32.tar.gz owned by root:root
    in /var/www/clients/client1/web22 there is a folder named backup created at 00:32, owned by web22:client1, this folder is empty

    should these folders not be linked in some way? either by a symbolic link, or a bind mounting in /etc/fstab like for the /var/log/ispconfig/httpd/<domain> /var/www/clients/<clientid>/<webid>/log entries?

    i configured the backup in the control panel whilst logged in as admin, not the user. could that be an issue? i wouldn't expect it to be, and if it is, i think it shouldn't be.
    i've configured another site for backups, as the user, but i want to let it run normally, so i won't know the result of that until tomorrow.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    That's all fine and as it should be, backups are stored in /var/backup, the backup folder in teh website is a empty directory where backups are made available on request. When a client wants to download a backup or restore it, then he can request that backup in ISPConfig on the backup tab of the website and ISPConfig will make this backup available then in his local website backup folder.
     
  3. nhybgtvfr

    nhybgtvfr Member

    ok,
    that explains the /var/www/<clients>/<clientid>/<webid>/backup folder being empty.

    but when i look at the backup tab for the website, both as the user and as admin, it doesn't list anything under existing backups, so i can't request the backup.
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Is this a single or multuserver system? Do you use the current ispconfig version 3.0.5.4p8?
     
  5. nhybgtvfr

    nhybgtvfr Member

    it's a multiserver system
    1 server running the control panel, no server services enabled
    1 server running a webserver (vps1), and db, as a private vm for 1 client
    1 server running a webserver (web1), for shared hosting, more to be added as we migrate more sites over.
    1 server running a db, all databases for sites on web1 are on this server, along with ispconfig databases for the cp, dns1, and itself
    1 server running mail, figured this would have heavy db access for user auth, so it's isponfig db is local.
    2 servers running dns, dns1 on same subnet as the other servers, ispconfig db on the db server.
    dns2 is geographically remote and runs it's ispconfig db on itself.
    it's ispconfig 3.0.5.4p8 on every server, os is ubuntu 14.04. php is 5.6.18 (php7.0.3 additional), db is percona 5.6
    also /var/www on web2 is an ext4 partition on another server, exported and mounted via nfs4, all shared hosting webservers will use this shared storage so sites can be migrated across servers or mirrored to multiple servers without having to move/copy any data files.
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    Then the slave is not avle to write the backup result into the master database. Run an ispconfig update on one of the slave nodes, does not matter which one, and choose to update permissions in the master database.
     
  7. nhybgtvfr

    nhybgtvfr Member

    ok, done that,
    is the existing backup supposed to show up now? after a web resync? after the next daily cron?
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    Existing backups wont show up, only new ones after next nightly cron run.
     
  9. nhybgtvfr

    nhybgtvfr Member

    cool, half expected that to be the case anyway.
    should i just delete the entry that does exist in the web_backup table in the webserver ispconfig db then?
    not looked at the links between tables, am i ok to reset the auto-increment value for that table back to 0?

    thanks.
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    You can delete that record.
     
  11. nhybgtvfr

    nhybgtvfr Member

    ok, that still doesn't work for the sites on web1, i set it up yesterday to do a backup of the site on vps1.
    i've checked that this morning and that is showing the database and website backup files in the control panel

    i'm gonna check the mysql user accounts and grants. but there's quite a lot of accounts on there now, which one should be making this change? one of the root accounts, ispconfig accounts, or ispcsrv* accounts?
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    the ispcsrv account of this slave tries to write to the backup table on the master.
     
  13. nhybgtvfr

    nhybgtvfr Member

    the grants look ok to me:
    web1:
    GRANT SELECT, INSERT, DELETE ON `ispconfig_CP`.`web_backup` TO 'ispcsrv6'@'web1.scl.co.uk'
    GRANT SELECT, INSERT, DELETE ON `ispconfig_CP`.`web_backup` TO 'ispcsrv6'@'10.10.12.101'
    vps1:
    GRANT SELECT, INSERT, DELETE ON `ispconfig_CP`.`web_backup` TO 'ispcsrv8'@'vps1.scl.co.uk'
    GRANT SELECT, INSERT, DELETE ON `ispconfig_CP`.`web_backup` TO 'ispcsrv8'@'10.10.12.201'

    i've also connected, from web1, using mysql -h 10.10.12.2 -u ispcsrv6 -p
    use ispconfig_CP;
    insert into web_backup (server_id, parent_domain_id, backup_type, backup_mode, tstamp, filename) VALUES ( 6,6,'web','rootgz','1458520567','test.tar.gz');

    which worked, i also deleted this new row using
    delete from web_backup where backup_id = '3';
    which also worked.
     
  14. nhybgtvfr

    nhybgtvfr Member

    hi, any further ideas on what can be causing this problem?
     
  15. nhybgtvfr

    nhybgtvfr Member

    i've been looking at this some more, and i still can't get it working properly.

    as stated before, on the vps, which is running both the website and the sites database, it's working,
    and in the ispconfig cron.log, around the time of the backups being done, i see a log entry warning about the use of passwords on the command line, which i'm assuming to be the script updating the web_backup table on the master database.

    for the sites where the backup isn't working, the webserver and database server are NOT the same server, the webserver is backing up the website files to /var/backup/web*/web<ID>_<DATE>_<TIME>.tar.gz, the database is backing up the databases to /var/backup/web<ID>_<customerID><dbname>_<date>_<time>.sql.gz
    could it be failing because the webfiles backup and db backup are happening on different servers and it's waiting for the other part to complete locally, or not getting notified the other server completed it's part?

    the relevent ispcsrv* account on the vps server where the backup works, and on the webserver and dbserver where the backup's don't work, all have insert, update and delete permissions to the master DB.

    alternatively, could the fact that the website files for the shared hosting webserver are actually on an NFS mount be the problem?
    i can see this in the ispconfig cron.logs on the webserver:
    00:32:28 UTC 2016 chattr: Inappropriate ioctl for device while reading flags on /var/www/clients/client1/web22
     
  16. nhybgtvfr

    nhybgtvfr Member

    ok.
    i think i've found where the problem is now.
    i think it's this line in cron_daily.php

    if($app->db->dbHost != $app->dbmaster->dbHost) $app->dbmaster->query($sql);

    i'm using a single central database server for all our systems, with only the mailserver having it's own local db instance, due to it's high expected authentication usage, and vps systems having their own db instances, since they wouldn't be very private otherwise.
    so checking with:
    var_dump ($app->db->dbHost);
    var_dump ($app->dbmaster->dbHost);
    returns string(10) "10.10.12.2" for both, so there is never an attempt to add the entry to the master database.

    the web server should be adding it to ispconfig_web1 and ispconfig_CP
    the database sever should be adding it to ispconfig_db and ispconfig_CP
    all 3 db's being on host 10.10.12.2
     
  17. till

    till Super Moderator Staff Member ISPConfig Developer

    An ISPConfig multiserver expects that each node runs at least, a minimal MySQl instance for fault tolerance. The connection should be done by hostname, not IP, so you might be able to fake the system by using a different hostname for the master connection from slave nodes.
     
  18. nhybgtvfr

    nhybgtvfr Member

    yep, i was hoping i could avoid the overhead of having mysql servers on everything, and since i was going with a large central db server, it seemed sensible to have them all on there.
    would have thought plenty of others would have gone for a setup like that as well.

    i've just added the hostname into the dbmaster_host field in config.inc.php and that seems to have fixed the problem for now.
    guess i should look at getting mysql server onto everything and migrating the db's to local instances.

    i've also noticed that when creating client's as a reseller, it doesn't pick up the correct next customer id field, similarly, then logging in as admin, to create another client, it still tries to use the customer id number that's just been used for the reseller's client.
    is the reseller's client id numbers supposed to be separate from the others, or part of the same sequence?
    currently the master db has the latest client with a client id of 22, customer number of 22, and the next autoindex shows 23, yet if i try to add another client it automatically fills in the customer number with c22, (c1 if logged in trying to add a client as a reseller)
    is this another anomaly caused by having all the databases on the same server? or is this a completely separate problem?
     
  19. till

    till Super Moderator Staff Member ISPConfig Developer

    ISPConfig uses a separate DB instance for each node to provide fault tolerance, we could have used a central DB as design but this would have caused ispconfig to be non fault tolerant and not scalable, wit the setup that we have chosen for ISPConfig, each node is able to operate on its own when any another node fails, even the master node can be shut down without affecting a slave server. You removed this fault tolerance with your setup so that your system is not more reliable than a single server system now.

    That's mots likely not related to your central databse issue
     

Share This Page