Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 3 > Plugins/Modules/Addons

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #111  
Old 10th May 2011, 01:25
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

I think that it should work, but maybe rsync is better for what you want (but you will not have an incremental backup)
Reply With Quote
The Following User Says Thank You to erosbk For This Useful Post:
cpaun (10th May 2011)
Sponsored Links
  #112  
Old 12th May 2011, 16:26
Croydon Croydon is offline
ISPConfig Developer
 
Join Date: Jul 2007
Location: Koblenz, Germany
Posts: 927
Thanks: 16
Thanked 261 Times in 208 Posts
Default

Hi, great script.
I would suggest using the --hex-blob option and the field and database name quoting when using mysqldump.
Reply With Quote
  #113  
Old 12th May 2011, 17:09
cpaun cpaun is offline
Junior Member
 
Join Date: Feb 2008
Posts: 18
Thanks: 3
Thanked 0 Times in 0 Posts
Default ISPC 2 backup and restore

Quote:
Originally Posted by Croydon View Post
Hi, great script.
I would suggest using the --hex-blob option and the field and database name quoting when using mysqldump.
Thank you for suggestion, do you have any HOWTO steps for that. I'm sure that it is many users will like to have an automatic backup and restore module.


Thanks,
Reply With Quote
  #114  
Old 12th May 2011, 17:40
Croydon Croydon is offline
ISPConfig Developer
 
Join Date: Jul 2007
Location: Koblenz, Germany
Posts: 927
Thanks: 16
Thanked 261 Times in 208 Posts
Default

Quote:
Originally Posted by cpaun View Post
Thank you for suggestion, do you have any HOWTO steps for that. I'm sure that it is many users will like to have an automatic backup and restore module.


Thanks,
I just meant the back-res script uses mysqldump without the --hex-blob and --quote-names (-Q) command line options.

So i would change
Code:
    ### Starting database dumps
    for i in `mysql -u$dbuser -p$dbpassword -Bse 'show databases'`; do
	log "Starting mysqldump $i"
	`mysqldump -u$dbuser -p$dbpassword $i --allow-keywords --comments=false --add-drop-table > $tmpdir/db-$i-$FDATE.sql`
	$TAR $ARG $BACKUPDIR/$MDATE/db-$i-$FDATE.tar.bz2 -C $tmpdir db-$i-$FDATE.sql
	rm -rf $tmpdir/db-$i-$FDATE.sql
	log "Dump OK. $i database saved OK!"
    done
to

Code:
    ### Starting database dumps
    for i in `mysql -u$dbuser -p$dbpassword -Bse 'show databases'`; do
	log "Starting mysqldump $i"
	`mysqldump -u$dbuser -p$dbpassword $i --allow-keywords --comments=false --add-drop-table --hex-blob --quote-names > $tmpdir/db-$i-$FDATE.sql`
	$TAR $ARG $BACKUPDIR/$MDATE/db-$i-$FDATE.tar.bz2 -C $tmpdir db-$i-$FDATE.sql
	rm -rf $tmpdir/db-$i-$FDATE.sql
	log "Dump OK. $i database saved OK!"
    done
Reply With Quote
  #115  
Old 14th May 2011, 10:14
Croydon Croydon is offline
ISPConfig Developer
 
Join Date: Jul 2007
Location: Koblenz, Germany
Posts: 927
Thanks: 16
Thanked 261 Times in 208 Posts
Default

I had a closer look at how the incremental directory backup is done here.

Code:
	    # If it's not the first day of the month we make incremental backup
	    if [ ! -e $tmpdir/full-backup$XX.lck ] ; then
		log "Starting daily backup for: $YX"
		NEWER="--newer $FDATE"
		$TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
		log "Daily backup for $YX done."
	    else
	    log "Lock file for $YX full backup exists!"
	    fi
I think this could lead to problems if (for any reason) one incremental run did not work. You could be missing files in the archives then, as the --newer flag is date-based.

Wouldn't it be a good idea to use the incremental flag of tar?

So instead of using
Code:
$TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
you could use
Code:
$TAR $ARG -g $BACKUPDIR/$MDATE/backup$XX.state $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
Of course you would have to add the -g $BACKUPDIR/$MDATE/backup$XX.state to the full backup command, too.

Last edited by Croydon; 14th May 2011 at 10:17.
Reply With Quote
  #116  
Old 17th May 2011, 13:58
go0ogl3 go0ogl3 is offline
Member
 
Join Date: Jul 2008
Posts: 73
Thanks: 5
Thanked 39 Times in 6 Posts
Exclamation tar minuses

Quote:
Originally Posted by Croydon View Post
I had a closer look at how the incremental directory backup is done here.

Code:
	    # If it's not the first day of the month we make incremental backup
	    if [ ! -e $tmpdir/full-backup$XX.lck ] ; then
		log "Starting daily backup for: $YX"
		NEWER="--newer $FDATE"
		$TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
		log "Daily backup for $YX done."
	    else
	    log "Lock file for $YX full backup exists!"
	    fi
I think this could lead to problems if (for any reason) one incremental run did not work. You could be missing files in the archives then, as the --newer flag is date-based.

Wouldn't it be a good idea to use the incremental flag of tar?

So instead of using
Code:
$TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
you could use
Code:
$TAR $ARG -g $BACKUPDIR/$MDATE/backup$XX.state $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
Of course you would have to add the -g $BACKUPDIR/$MDATE/backup$XX.state to the full backup command, too.


Hello,

In my case, the tar incremental backup is not a very good solution. In the case you loose a incremental backup with my script you can restore the others. See below what's happening with tar incremental backups.

With tar "if you plan to create more ‘level 1’ backups, it is necessary to create a working copy of the snapshot file before running tar". With the script this is not the case.
The second minus from tar is "Incremental dumps depend crucially on time stamps, so the results are unreliable if you modify a file's time stamps during dumping". My script does not suffer from this and it was created partially because of this tar problem.

You can read here more: http://www.gnu.org/software/automake...tal-Dumps.html
The quoted examples are from the link above.
Reply With Quote
  #117  
Old 17th May 2011, 14:12
go0ogl3 go0ogl3 is offline
Member
 
Join Date: Jul 2008
Posts: 73
Thanks: 5
Thanked 39 Times in 6 Posts
Smile to be changed

Quote:
Originally Posted by Croydon View Post
I just meant the back-res script uses mysqldump without the --hex-blob and --quote-names (-Q) command line options.

So i would change
Code:
    ### Starting database dumps
    for i in `mysql -u$dbuser -p$dbpassword -Bse 'show databases'`; do
	log "Starting mysqldump $i"
	`mysqldump -u$dbuser -p$dbpassword $i --allow-keywords --comments=false --add-drop-table > $tmpdir/db-$i-$FDATE.sql`
	$TAR $ARG $BACKUPDIR/$MDATE/db-$i-$FDATE.tar.bz2 -C $tmpdir db-$i-$FDATE.sql
	rm -rf $tmpdir/db-$i-$FDATE.sql
	log "Dump OK. $i database saved OK!"
    done
to

Code:
    ### Starting database dumps
    for i in `mysql -u$dbuser -p$dbpassword -Bse 'show databases'`; do
	log "Starting mysqldump $i"
	`mysqldump -u$dbuser -p$dbpassword $i --allow-keywords --comments=false --add-drop-table --hex-blob --quote-names > $tmpdir/db-$i-$FDATE.sql`
	$TAR $ARG $BACKUPDIR/$MDATE/db-$i-$FDATE.tar.bz2 -C $tmpdir db-$i-$FDATE.sql
	rm -rf $tmpdir/db-$i-$FDATE.sql
	log "Dump OK. $i database saved OK!"
    done
Hello,

In the next version of the script perhaps I might fix this and some other small issues.
Reply With Quote
  #118  
Old 17th May 2011, 18:15
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Hello!

I have a problem with this line... all is working ok but... in this line I get an error when trying to restore...

Code:
		mysql -u$dbuser -p$dbpassword $rdb < $TAR -xvjp $BACKUPDIR/$YDATE-$MDATE/$i
the $TAR variable is ok (echo $TAR shows its value), but, if I run:

echo mysql -u$dbuser -p$dbpassword $rdb < $TAR -xvjp $BACKUPDIR/$YDATE-$MDATE/$i

I get: mysql -uuser -ppassword notremember < -xvjp /mibackupdir/date/blabla

It is like mysql were eating the value of $TAR!!!

I write ANOTHER $TAR in the sentence... (... < $TAR $TAR ... ) and the echo shows it correctly:

mysql -uuser -ppassword notremember < /TARPATH -xvjp /mibackupdir/date/blabla

but I get an error =(

I am using Debian, using bash as shell... could you help me? I think that a work around could to untar in a tmp directory, and next do the dump... (this workaround would be great... I didn't have time to modify it...)

Thanks for this perfect and usefull script.
Reply With Quote
  #119  
Old 19th May 2011, 22:10
go0ogl3 go0ogl3 is offline
Member
 
Join Date: Jul 2008
Posts: 73
Thanks: 5
Thanked 39 Times in 6 Posts
Default ?

Quote:
Originally Posted by erosbk View Post
Hello!

I have a problem with this line... all is working ok but... in this line I get an error when trying to restore...

Code:
		mysql -u$dbuser -p$dbpassword $rdb < $TAR -xvjp $BACKUPDIR/$YDATE-$MDATE/$i
the $TAR variable is ok (echo $TAR shows its value), but, if I run:

echo mysql -u$dbuser -p$dbpassword $rdb < $TAR -xvjp $BACKUPDIR/$YDATE-$MDATE/$i

I get: mysql -uuser -ppassword notremember < -xvjp /mibackupdir/date/blabla

It is like mysql were eating the value of $TAR!!!

I write ANOTHER $TAR in the sentence... (... < $TAR $TAR ... ) and the echo shows it correctly:

mysql -uuser -ppassword notremember < /TARPATH -xvjp /mibackupdir/date/blabla

but I get an error =(

I am using Debian, using bash as shell... could you help me? I think that a work around could to untar in a tmp directory, and next do the dump... (this workaround would be great... I didn't have time to modify it...)

Thanks for this perfect and usefull script.
I've have this script working in multiple debian machines, different stable, but never had any issues...
Right now I can't imagine what is happening in your particular case, but please if you find out what was wrong, or if you find a solution to your case, or maybe how can I reproduce the error, please write here.

By the way what's your Debian version? What packages are installed on your computer? Did you try to duplicate the install onto a virtual machine (virtualbox or vmware) to check if the problem persist?

Thank you for your time!
Reply With Quote
  #120  
Old 22nd May 2011, 11:48
StefanoVanzo StefanoVanzo is offline
Junior Member
 
Join Date: Mar 2008
Posts: 4
Thanks: 0
Thanked 0 Times in 0 Posts
 
Default

Quote:
Originally Posted by go0ogl3 View Post
Hello again!

I think you use the latest mc with the latest settings and you just used some mouse copy and paste for the script... so you copied a "point" instead of a space. And here is the error...
Quote:
Originally Posted by cyrus1977 View Post
Warning: the directories lines contains an error:

DIRECTORIES="/bin /boot /etc /home /lib /lib64 /root.
/sbin /usr /var /www"

should read

DIRECTORIES="/bin /boot /etc /home /lib /lib64 /root
/sbin /usr /var /www"

attention: /root. has been changed to /root
I think cyrus1997 is right. Can you please check.
For me in the script there is a dot (.) after /root and this prevents from backing up the /root directory.
If I am wrong, please tell me why.

working with IspConfig3 on Debian Squeeze 64bit
thank you for your work and you time

Last edited by StefanoVanzo; 22nd May 2011 at 11:50.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
freebsd 7, samba 3, domain controller alexdimarco Suggest HOWTO 6 5th November 2010 16:54
how to restore Ispconfig from backup db na0lb Installation/Configuration 4 24th September 2008 08:51
Restore backup ISPConfig leeeonardo General 4 18th April 2007 20:21
Restore ispconfig complete backup statiic General 1 15th January 2007 10:32
Backup restore on ispconfig misterm Installation/Configuration 0 18th December 2005 23:32


All times are GMT +2. The time now is 01:52.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.