Upgrade to ISPConfig 3.1 no backup

Discussion in 'General' started by bdjens, Oct 3, 2016.

  1. bdjens

    bdjens New Member

    Hi, after updating to 3.1, all backup do not work, not made.
    I deleted the previous ones in / var / backup / webxxx and still appear on the list of backup client.
    I've reconfigured the backup daily and does nothing.
    Could someone please help me?
     
  2. bdjens

    bdjens New Member

    help please
    I do not work the backup of the website or emails after upgrading ... why?
    where I can find the log files to know what happens?
     
    Last edited: Oct 4, 2016
  3. chico11mbit

    chico11mbit Member HowtoForge Supporter

    here the same on a backup mount. also i cannot delete old backups.

    in the ispconfig.log i found for deleting a existing old backup:

    10.11.2016-02:04 - DEBUG - Processed datalog_id 5971
    10.11.2016-02:04 - DEBUG - Calling function 'backup_action' from plugin 'backup_plugin' raised by action 'backup_delete'.
    10.11.2016-02:04 - DEBUG - Backup directory not ready.

    hmm, when i disable "Backup directory is a mount?" it works. But it is a NFS volume on a NAS which worked perfectly in 3.0.5.
     
    Last edited: Nov 10, 2016
  4. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    If you enable "Backup directory is a mount", check /usr/local/ispconfig/server/scripts/backup_dir_mount.sh and the backup-dir must be empty if the storage is unmounted.
     
  5. chico11mbit

    chico11mbit Member HowtoForge Supporter

    Hi Florian,
    i haven't this file in this directory. The only files i have in this directory are:

    create_daily_nginx_access_logs.sh
    create_jailkit_chroot.sh
    create_jailkit_programs.sh
    create_jailkit_user.sh
    ispconfig_htaccess.php
    ispconfig_patch
    ispconfig_update.php
    ispconfig_update.sh
    update_from_dev.sh
    update_from_dev_stable.sh
    update_from_svn.sh
    update_from_tgz.sh
    update_stable.sh
    vlogger

    update: ok. i have added that file and chmod -x it. In this file is:
    mount /mnt/"the NFS device which is set in fstab"
    and also this tree is empty when i unmount it.

    is this ok?

    2. Question:
    When i disable "backup dir is a mount" in ispconfig i can delete the backups which are created before update to 3.1. But no new backup is created at the time i set in ispconfig for the Backup Time. Also there is no error in /var/log/ispconfig/ispconfig.log when this is set to debug mode.
     
    Last edited: Nov 10, 2016
  6. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    Check the system-log in ispconfig and check your loglevel.
    You can also create a script called backup_dir_umount.sh to unmount the storage after the backups
    Both must be owned by root.
    i use:
    Code:
    #!/bin/sh
    sshfs [email protected]:/backups /var/backup.mnt
    
     
  7. chico11mbit

    chico11mbit Member HowtoForge Supporter

    The log level is "debug".
    The NFS mount is mounted at boot and should not be unmounted because other programs are using this device too.
    i have not installed sshfs, so is the command i used in #5 ok? ( mount /mnt/"the NFS device which is set in fstab")

    btw. Also no email accounts are backuped. Same symptom as when i try to backup web folders and databases.
     
  8. chico11mbit

    chico11mbit Member HowtoForge Supporter

    Hi Florian,
    another info. Perhaps it helps.
    i installed in the prior version of ISPCONFIG 3.0.5.4p9 your mail backup plugin from:
    https://blog.schaal-24.de/ispconfig/ispconfig-mailbackup/
    and in /usr/local/ispconfig/server is a file called mailbackup.php.inc.
    is this ok or is the prior installation of your mail backup plugin making errors in the 3.1.1. installation?
     
  9. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    This file should not be used with ispconfig 3.1. the cron-jobs are split to several classes / files. mailbackup.php.inc. was called from cron_daily and cron_dalily was removed from 3.1 (and the crontab)
     
  10. Jesse Norell

    Jesse Norell Active Member

    I'm working on mount/unmount scripts (using sshfs + fuse, I'll post them when ready) and they work fine from cli, but I find the backup mount point is not getting unmounted, at least after clicking "Download" to download a client's backup. The mount is working correctly, and the backup file is copied to the right spot for the client, just the unmount doesn't happen - is this something I have in my environment, or is that a bug? Also, does it get unmounted after creating backups each night? (I had the unmount script misnamed last night, but it's correct now, so I should know tomorrow.)
     
  11. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    Code:
    #!/bin/sh
    sshfs [email protected]:/backups /var/backup.mnt
    
    and

    Code:
    #!/bin/sh
    umont /var/backup.mnt
    
     
  12. Jesse Norell

    Jesse Norell Active Member

    After further testing I created https://git.ispconfig.org/ispconfig/ispconfig3/issues/4346

    For the curious, the unmount does happen when nightly backups are created.
     
  13. Jesse Norell

    Jesse Norell Active Member

    If useful to anyone, here's a mount/unmount script that uses sshfs + encfs, and optionally stores the encryption passphrase on a mandos server. Easy to adapt to CryFS if you don't like encfs.

    /usr/local/ispconfig/server/scripts/backup_dir_mount.sh:
    Code:
    #!/bin/bash
    
    # backup_dir_mount.sh - called by ispconfig
    # to mount the backup directory prior to backing up
    #
    # This uses sshfs + encfs + mandos
    # to securely mount a remote server (sshfs),
    # and encrypt the remote data (encfs).
    # We store the encryption key on the mandos server.
    
    # Final desination where backups will be made, files within it are unencrypted:
    BACKUP_DIR=/var/ispc_backup_mnt
    
    # This file must exist once encfs filesystem is fully accessible
    CHECK_FILE="${BACKUP_DIR}/README.failsafe"
    
    # We sshfs mount this directory, and files within it are encrypted:
    SSHFS_MNT_DIR=/var/.ispc_backup_sshfs_mnt
    
    # Remote user/server and directory to mount
    SSH_USER="[email protected]"
    SSHFS_DIR="/mnt/ispconfig_client_backups/`hostname -s`"
    SSHFS_SOURCE="${SSH_USER}:${SSHFS_DIR}"
    
    # Encryption key or 'mandos' to call mandos-client
    KEY=mandos
    
    # path to mandos-client
    MANDOS_CLIENT=
    MANDOS_TIMEOUT=60
    PUBKEY=/etc/keys/mandos/pubkey.txt
    SECKEY=/etc/keys/mandos/seckey.txt
    
    function die() {
        echo "${@}" 1>&2
        exit 1
    }
    
    if [ ! -d "${SSHFS_MNT_DIR}" ]
    then
        mkdir -m 770 -p "${SSHFS_MNT_DIR}" || die "Unable to create sshfs mount dir ${SSHFS_MNT_DIR}"
    fi
    
    test -d "${SSHFS_MNT_DIR}" || die "Directory does not exist: ${SSHFS_MNT_DIR}"
    test -d "${BACKUP_DIR}" || die "Directory does not exist: ${BACKUP_DIR}"
    
    ssh ${SSH_USER} test -d ${SSHFS_DIR}
    if [ $? -ne 0 ]
    then
        echo "Error: Remote directory does not exist.  Try creating it with:
    
        ssh ${SSH_USER} mkdir -p ${SSHFS_DIR}
    "
        die
    fi
    
    which sshfs > /dev/null || die "sshfs: command not found"
    which encfs > /dev/null || die "encfs: command not found"
    
    if [ "${KEY}" == "mandos" ]
    then
        if [ -z "${MANDOS_CLIENT}" ] || [ ! -f "${MANDOS_CLIENT}" ]; then
          MANDOS_CLIENT=$(which mandos-client /usr/lib/{,*/}mandos/plugins.d/mandos-client | head -1)
          if [ ! -f "${MANDOS_CLIENT}" ]; then
            die "mandos-client not found"
          fi
        fi
    
        test -f "${PUBKEY}" || die "Error: pubkey (${PUBKEY}) not found.  Need to run mandos-keygen?"
        test -f "${SECKEY}" || die "Error: seckey (${SECKEY}) not found.  Need to run mandos-keygen?"
    
        KEY=$(timeout ${MANDOS_TIMEOUT} ${MANDOS_CLIENT} --pubkey=${PUBKEY} --seckey=${SECKEY})
    fi
    
    test -z "${KEY}" && die "Could not obtain encryption key"
    
    if [ ! -f "${SSHFS_MNT_DIR}/.encfs6.xml" ]
    then
        # sshfs mount remote server
        sshfs -o uid=0,gid=0,nosuid,nodev ${SSHFS_SOURCE} ${SSHFS_MNT_DIR} || die "Error: sshfs mount failed"
    fi
    
    if [ ! -f "${SSHFS_MNT_DIR}/.encfs6.xml" ]
    then
        echo "EncFS does not appear to be created on the remotely mounted directory." 1>&2
        echo "First create it with:  encfs -o allow_other ${SSHFS_MNT_DIR} ${BACKUP_DIR}" 1>&2
        echo "Then unmount and try again:  fusermount -u ${BACKUP_DIR}; fusermount -u ${SSHFS_MNT_DIR}" 1>&2
        die
    fi
    
    if [ ! -f "${CHECK_FILE}" ]
    then
        # encfs mount remote mount
        encfs --stdinpass -o nonempty,nosuid,nodev ${SSHFS_MNT_DIR} ${BACKUP_DIR}  <<< "${KEY}" || die "Error: encfs mount failed"
    fi
    
    test -f "${CHECK_FILE}" || die "Error: failsafe check file not found (${CHECK_FILE})"
    
    exit 0
    
    /usr/local/ispconfig/server/scripts/backup_dir_umount.sh:
    Code:
    #!/bin/bash
    
    # backup_dir_unmount.sh - called by ispconfig
    # to unmount the backup directory after backing up
    #
    # The mount uses sshfs + encfs + mandos
    # to securely mount a remote server (sshfs),
    # and encrypt the remote data (encfs).
    # As such, the unmount is a simple fusermount -u
    
    # Final desination where backups will be made, files within it are unencrypted:
    BACKUP_DIR=/var/ispc_backup_mnt
    
    # This file must exist once encfs filesystem is fully accessible
    CHECK_FILE="${BACKUP_DIR}/README.failsafe"
    
    # We sshfs mount this directory, and files within it are encrypted:
    SSHFS_MNT_DIR=/var/.ispc_backup_sshfs_mnt
    
    function die() {
        echo "${@}" 1>&2
        exit 1
    }
    
    test -d "${BACKUP_DIR}" || die "Directory does not exist: ${BACKUP_DIR}"
    test -d "${SSHFS_MNT_DIR}" || die "Directory does not exist: ${SSHFS_MNT_DIR}"
    
    test -f "${CHECK_FILE}" || die "Error: encfs appears to be not mounted, failsafe check file not found (${CHECK_FILE})"
    test -f "${CHECK_FILE}" && (fusermount -u "${BACKUP_DIR}" || die "encfs fusermount unmount failed")
    test -f "${CHECK_FILE}" && die "Error: encfs fusermount unmount appears to have failed"
    
    test -f "${SSHFS_MNT_DIR}/.encfs6.xml" && (fusermount -u "${SSHFS_MNT_DIR}" || die "sshfs fusermount unmount failed")
    test -f "${SSHFS_MNT_DIR}/.encfs6.xml" && die "Error: sshfs fusermount unmount appears to have failed"
    
    exit 0
    
    Make those both executable, and remember to edit the Server Config to set the backup directory (and/or change the scripts to match) and enable 'Backup directory is a mount?' checkbox.
     

Share This Page