Another backup solution

Discussion in 'Tips/Tricks/Mods' started by ncoc.nl, Mar 16, 2011.

  1. ncoc.nl

    ncoc.nl New Member

    Another FTP backup solution

    Hello everyone,

    Because of not finding a backup solution that fit my needs i've take some pieces from other scripts added some of my own and rewrite it to this one.

    How it works:
    The script makes a backup of the MySQL database in separate files (each DB has it own file) every 4 hours by using a cronjob, these files are stored in a local directory.

    When finished the DB backup and the ISPconfig (Website) backup are copied to a FTP server, to prevent unneccessary load and bandwidth use the backup of the websites is copied once a day during the night (ISPconfig makes 1 backup at 0:30 so extra copies are not needed)

    For this script the backup option in ISPconfig must be set.

    Do NOT forget to change the < > values to fit your needs

    A little manual is in top of the script.

    Ronald
     

    Attached Files:

    Last edited: Jun 23, 2011
  2. ncoc.nl

    ncoc.nl New Member

    Still work in progress, I did some modifications.

    What has changed?
    The MySQL backup will now keep 7 day en 3 sunday backups, then the oldest will be deleted in the local backup.
    The backup directory structure is changed, it will now create a time directory inside the date directory.
    Added echo's for screen output when running the script in screen.

    TODO:

    • Purge option on remote FTP server - Delete after 7 days, keep sunday for 3 weeks
    • Purge option on remote FTP server - Delete sunday backup after 22 days
    • Create exclusion "if directory exist"
    • Include ISPconfig and all files needed for a reatore in case of a crash
    The script is as/is, it does work but I couldn't test the rotation for the 22 days.

    Ronald
     

    Attached Files:

  3. giganthost

    giganthost New Member

    good script

    Hello,
    thanks for this great script.

    Please be possible in the script set to be compressed database?
     
  4. ncoc.nl

    ncoc.nl New Member

    The dump is done with mysqldump, you can compress the DB dir by using:

    cd /var/backup
    tar -pczf <your_filename>.tar.gz <your_directory_name>/

    If neccessary you can delete the dir. afterwards, it will be created with the next backup.

    Ronald
     
  5. Croydon

    Croydon HowtoForge Supporter

    As I mentioned in a different thread, I would suggest adding the --hex-blob and --quote-names command line options to the mysqldump command.

    If you have binary blob columns in your database you might run into trouble restoring it from a dump without hex-blob.
     
  6. ncoc.nl

    ncoc.nl New Member

    Hi everyone,

    For those who are interested, i did a complete rewrite of the wdbackup.sh and create 2 backupscripts, 1 for an external (your own?) FTP server and 1 for an internal FTP server (hired from or provided by the server hoster)

    Short description:

    Using an internal FTP backupserver.
    This script will copy the websites backup files at 02.00H, the MySQL databases every 4 hours starting at 02.00H and the ISPconfig website and SQL database at 14.00H and 22:00H to an internal FTP server.
    The MySQL backup databases on the local server will be purged every 7 days except the sunday backup, this database will be purged every 4 weeks.

    Using an external FTP backupserver.
    This script will copy the websites at 02.00H, the MySQL databases every 4 hours starting at 02.00H and the ISPconfig website and SQL database at 14.00H and 22:00H to a external FTP server of your choice. Every MySQL backup is overwritten after 24 hours, the websites backup is overwritten after 2 days. The MySQL backup databases on the local server will be purged every 7 days exept the sunday backup, this database will be purged every 4 weeks.

    I've used the article http://www.howtoforge.com/forums/showthread.php?t=52466 which describes a standalone FTP server with a webinterface to setup my own backup solution.


    The scripts are located in the first post
    More information is in top of both scripts.

    Ronald
     
  7. nokia80

    nokia80 HowtoForge Supporter

    work well alone bakup ISPConfig files does not work.

    I use the wrong script?


    thanks
     
  8. ncoc.nl

    ncoc.nl New Member

    The script is written for debian etch withISPconfig 3.x, possibly you're using a different distro?

    Ronald
     
  9. nokia80

    nokia80 HowtoForge Supporter

    yes i use debian on the mail log i have this mesage:

    ISPconfig: No Backup scheduled.

    These are the most recent backup files.


    thanks
     
  10. ncoc.nl

    ncoc.nl New Member

    The backup of ISPconfig runs at 14:00H and 22:00H, in the notification e-mail backups which are not scheduled to run are mentioned as: No Backup scheduled.

    I've found a little "bug" in the backupscript, the backup of ISPconfig is called twice in the script, when it runs e.g. at 14:00 hours the second part which is scheduled at 22:00 will give the message: No Backup scheduled. (and at 22:00 it will give the same message)

    Working on it.

    Ronald
     
  11. neobcn

    neobcn New Member

    Howw to insytall in 3.0.3 and Debian 6??
     
  12. ncoc.nl

    ncoc.nl New Member

    I don't know, I run several Debian 5 Lenny servers, possibly it will run on Debian 6 as well.

    Maybe someone has tested the script already on Debian 6

    Ronald
     
  13. versatile

    versatile New Member

    Yes, it's works perfectly on Debian Squeeze but don't forget to reconfigure dash otherwise you have a error with time settings.

    Ronald nice script :)
     
  14. neobcn

    neobcn New Member

    Please how too configure i novice :-(
     
  15. versatile

    versatile New Member

    If you have a error like [ 92 : unexpected error ]. The problem maybe comes from Dash. In debian /bin/sh is a symlink to /bin/dash, however we need /bin/bash, not /bin/dash. But if you have ISPconfig on your server you have reconfigure Dash.( If you don't do this, the ISPConfig installation fail).

    If you have a error run this command in shell

    #bash -x your_script

    If it's works run this command in shell

    #dpkg-reconfigure dash

    Use dash as the default system shell (/bin/sh)? <-- No

    A other solution is to change in the script the first line #!/bin/sh to #!/bin/bash

    If you would like to test the script now ( all backups) at 21:25 and not at 21H you can change this line

    TIME=$(date +"%HH")

    to

    TIME=$(date +"%H:%M")

    you can change

    TIMEWEBBU="21H"
    TIMEISPBU1="21H"
    TIMEISPBU2="18H"

    to

    TIMEWEBBU="21:25"
    TIMEISPBU1="21:25"
    TIMEISPBU2="18:23"

    After just run at 21:25

    #./your_script

    to check if all backups work.
     
    Last edited: Jul 27, 2011
  16. nokia80

    nokia80 HowtoForge Supporter

    hi

    on debian squeezy i have this error

    ./wdbackup.sh: line 228: -u: command not found
    ./wdbackup.sh: line 228: -u: command not found
    ./wdbackup.sh: line 228: -u: command not found
    ./wdbackup.sh: line 228: -u: command not found
    ./wdbackup.sh: line 228: -u: command not found


    ??

    thanks
     
  17. falko

    falko Super Moderator

    What's in line 228 of wdbackup.sh?
     
  18. ncoc.nl

    ncoc.nl New Member

    Nokia80,

    I've downloaded the wdbackup.sh but theres no line 228, it goes up to line 125?
    In the other 2 scripts there's no -u present on the mentioned line, did you make zome modifications yourself?

    Ronald
     
    Last edited: Dec 20, 2011
  19. ncoc.nl

    ncoc.nl New Member

    Update

    For some reason I can't edit my post so I reply myself :)

    I did some modification of the original script:

    wdbackup-1: backup to FTP once a day
    wdbackup-6: backup the database 6 times a day, ISPconfig twice a day and the websites once a day.

    Also the script checks the (local) diskspace and when running low it wil generate a e-mail which you can send to an second e-mail adres.

    Al the info of the script is included in the header

    Ronald
     

    Attached Files:

  20. nokia80

    nokia80 HowtoForge Supporter

    there is somting wrong in the new script when i run it

    a lot of error

    can create directory
     

Share This Page