ISPConfig 3 - back-res a Backup and Restore script

[erosbk]

I think that it should work, but maybe rsync is better for what you want (but you will not have an incremental backup)

[Croydon]

Hi, great script.
I would suggest using the --hex-blob option and the field and database name quoting when using mysqldump.

[cpaun]

Originally Posted by Croydon

Hi, great script.
I would suggest using the --hex-blob option and the field and database name quoting when using mysqldump.

Thank you for suggestion, do you have any HOWTO steps for that. I'm sure that it is many users will like to have an automatic backup and restore module.


Thanks,

[Croydon]

Originally Posted by cpaun

Thank you for suggestion, do you have any HOWTO steps for that. I'm sure that it is many users will like to have an automatic backup and restore module.


Thanks,

I just meant the back-res script uses mysqldump without the --hex-blob and --quote-names (-Q) command line options.

So i would change

    ### Starting database dumps
    for i in `mysql -u$dbuser -p$dbpassword -Bse 'show databases'`; do
	log "Starting mysqldump $i"
	`mysqldump -u$dbuser -p$dbpassword $i --allow-keywords --comments=false --add-drop-table > $tmpdir/db-$i-$FDATE.sql`
	$TAR $ARG $BACKUPDIR/$MDATE/db-$i-$FDATE.tar.bz2 -C $tmpdir db-$i-$FDATE.sql
	rm -rf $tmpdir/db-$i-$FDATE.sql
	log "Dump OK. $i database saved OK!"
    done

to

    ### Starting database dumps
    for i in `mysql -u$dbuser -p$dbpassword -Bse 'show databases'`; do
	log "Starting mysqldump $i"
	`mysqldump -u$dbuser -p$dbpassword $i --allow-keywords --comments=false --add-drop-table --hex-blob --quote-names > $tmpdir/db-$i-$FDATE.sql`
	$TAR $ARG $BACKUPDIR/$MDATE/db-$i-$FDATE.tar.bz2 -C $tmpdir db-$i-$FDATE.sql
	rm -rf $tmpdir/db-$i-$FDATE.sql
	log "Dump OK. $i database saved OK!"
    done

[Croydon]

I had a closer look at how the incremental directory backup is done here.

	    # If it's not the first day of the month we make incremental backup
	    if [ ! -e $tmpdir/full-backup$XX.lck ] ; then
		log "Starting daily backup for: $YX"
		NEWER="--newer $FDATE"
		$TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
		log "Daily backup for $YX done."
	    else
	    log "Lock file for $YX full backup exists!"
	    fi

I think this could lead to problems if (for any reason) one incremental run did not work. You could be missing files in the archives then, as the --newer flag is date-based.

Wouldn't it be a good idea to use the incremental flag of tar?

So instead of using

$TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded

you could use

$TAR $ARG -g $BACKUPDIR/$MDATE/backup$XX.state $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded

Of course you would have to add the -g $BACKUPDIR/$MDATE/backup$XX.state to the full backup command, too.

[go0ogl3]

Originally Posted by Croydon

I had a closer look at how the incremental directory backup is done here.

	    # If it's not the first day of the month we make incremental backup
	    if [ ! -e $tmpdir/full-backup$XX.lck ] ; then
		log "Starting daily backup for: $YX"
		NEWER="--newer $FDATE"
		$TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
		log "Daily backup for $YX done."
	    else
	    log "Lock file for $YX full backup exists!"
	    fi

I think this could lead to problems if (for any reason) one incremental run did not work. You could be missing files in the archives then, as the --newer flag is date-based.

Wouldn't it be a good idea to use the incremental flag of tar?

So instead of using

$TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded

you could use

$TAR $ARG -g $BACKUPDIR/$MDATE/backup$XX.state $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded

Of course you would have to add the -g $BACKUPDIR/$MDATE/backup$XX.state to the full backup command, too.

Hello,

In my case, the tar incremental backup is not a very good solution. In the case you loose a incremental backup with my script you can restore the others. See below what's happening with tar incremental backups.

With tar "if you plan to create more ‘level 1’ backups, it is necessary to create a working copy of the snapshot file before running tar". With the script this is not the case.
The second minus from tar is "Incremental dumps depend crucially on time stamps, so the results are unreliable if you modify a file's time stamps during dumping". My script does not suffer from this and it was created partially because of this tar problem.

You can read here more: http://www.gnu.org/software/automake...tal-Dumps.html
The quoted examples are from the link above.

[go0ogl3]

Originally Posted by Croydon

I just meant the back-res script uses mysqldump without the --hex-blob and --quote-names (-Q) command line options.

So i would change

    ### Starting database dumps
    for i in `mysql -u$dbuser -p$dbpassword -Bse 'show databases'`; do
	log "Starting mysqldump $i"
	`mysqldump -u$dbuser -p$dbpassword $i --allow-keywords --comments=false --add-drop-table > $tmpdir/db-$i-$FDATE.sql`
	$TAR $ARG $BACKUPDIR/$MDATE/db-$i-$FDATE.tar.bz2 -C $tmpdir db-$i-$FDATE.sql
	rm -rf $tmpdir/db-$i-$FDATE.sql
	log "Dump OK. $i database saved OK!"
    done

to

    ### Starting database dumps
    for i in `mysql -u$dbuser -p$dbpassword -Bse 'show databases'`; do
	log "Starting mysqldump $i"
	`mysqldump -u$dbuser -p$dbpassword $i --allow-keywords --comments=false --add-drop-table --hex-blob --quote-names > $tmpdir/db-$i-$FDATE.sql`
	$TAR $ARG $BACKUPDIR/$MDATE/db-$i-$FDATE.tar.bz2 -C $tmpdir db-$i-$FDATE.sql
	rm -rf $tmpdir/db-$i-$FDATE.sql
	log "Dump OK. $i database saved OK!"
    done

Hello,

In the next version of the script perhaps I might fix this and some other small issues.

[erosbk]

Hello!

I have a problem with this line... all is working ok but... in this line I get an error when trying to restore...

		mysql -u$dbuser -p$dbpassword $rdb < $TAR -xvjp $BACKUPDIR/$YDATE-$MDATE/$i

the $TAR variable is ok (echo $TAR shows its value), but, if I run:

echo mysql -u$dbuser -p$dbpassword $rdb < $TAR -xvjp $BACKUPDIR/$YDATE-$MDATE/$i

I get: mysql -uuser -ppassword notremember < -xvjp /mibackupdir/date/blabla

It is like mysql were eating the value of $TAR!!!

I write ANOTHER $TAR in the sentence... (... < $TAR $TAR ... ) and the echo shows it correctly:

mysql -uuser -ppassword notremember < /TARPATH -xvjp /mibackupdir/date/blabla

but I get an error =(

I am using Debian, using bash as shell... could you help me? I think that a work around could to untar in a tmp directory, and next do the dump... (this workaround would be great... I didn't have time to modify it...)

Thanks for this perfect and usefull script.

[go0ogl3]

Originally Posted by erosbk

Hello!

I have a problem with this line... all is working ok but... in this line I get an error when trying to restore...

		mysql -u$dbuser -p$dbpassword $rdb < $TAR -xvjp $BACKUPDIR/$YDATE-$MDATE/$i

the $TAR variable is ok (echo $TAR shows its value), but, if I run:

echo mysql -u$dbuser -p$dbpassword $rdb < $TAR -xvjp $BACKUPDIR/$YDATE-$MDATE/$i

I get: mysql -uuser -ppassword notremember < -xvjp /mibackupdir/date/blabla

It is like mysql were eating the value of $TAR!!!

I write ANOTHER $TAR in the sentence... (... < $TAR $TAR ... ) and the echo shows it correctly:

mysql -uuser -ppassword notremember < /TARPATH -xvjp /mibackupdir/date/blabla

but I get an error =(

I am using Debian, using bash as shell... could you help me? I think that a work around could to untar in a tmp directory, and next do the dump... (this workaround would be great... I didn't have time to modify it...)

Thanks for this perfect and usefull script.

I've have this script working in multiple debian machines, different stable, but never had any issues...
Right now I can't imagine what is happening in your particular case, but please if you find out what was wrong, or if you find a solution to your case, or maybe how can I reproduce the error, please write here.

By the way what's your Debian version? What packages are installed on your computer? Did you try to duplicate the install onto a virtual machine (virtualbox or vmware) to check if the problem persist?

Thank you for your time!

[StefanoVanzo]

Originally Posted by go0ogl3

Hello again!

I think you use the latest mc with the latest settings and you just used some mouse copy and paste for the script... so you copied a "point" instead of a space. And here is the error...

Originally Posted by cyrus1977

Warning: the directories lines contains an error:

DIRECTORIES="/bin /boot /etc /home /lib /lib64 /root.
/sbin /usr /var /www"

should read

DIRECTORIES="/bin /boot /etc /home /lib /lib64 /root
/sbin /usr /var /www"

attention: /root. has been changed to /root

I think cyrus1997 is right. Can you please check.
For me in the script there is a dot (.) after /root and this prevents from backing up the /root directory.
If I am wrong, please tell me why.

working with IspConfig3 on Debian Squeeze 64bit
thank you for your work and you time