Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 3 > Plugins/Modules/Addons

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #131  
Old 23rd August 2011, 19:52
il.manuel il.manuel is offline
Junior Member
 
Join Date: Aug 2011
Posts: 10
Thanks: 2
Thanked 1 Time in 1 Post
Default

hi google and thx for your great job!

i have two problem related on server rather than script.

my scenario: a VPS and a ftp backup server on remote machine with a 10GB quota on it.

i have mounted the remote ftp server using curlftpfs (http://curlftpfs.sourceforge.net/) and everything seems to be ok.

first problem: df show me the entire disk space on remote server (7,5TB !) and not my quota (10GB). max percentage calculation have no way to run well and the ftp space will end without a check possibility.

second problem: i thgink that curlftpfs not support append to file ('>>'). i got a lot of this error

Code:
./backupres: line 236: /backup/<<HOSTNAME>>/log/backup.log: Operation not supported
when he try to update logfile.

I think for an alternative, but i can't find a solution (manually deleting old backups!!!)

third problem

he not send me an email at the end of script. /tmp/tmpbck/maildata exist and he contains every log entry.

Last edited by il.manuel; 23rd August 2011 at 20:15.
Reply With Quote
Sponsored Links
  #132  
Old 24th August 2011, 00:51
flyingscubakev flyingscubakev is offline
Junior Member
 
Join Date: Feb 2009
Location: Winnipeg, MB Canada
Posts: 29
Thanks: 1
Thanked 0 Times in 0 Posts
Default

Just a quick question from a relative newbie.
I've followed the perfect server - ispconfig3 on opesuse and everything is working, I've now loaded this backup script and it seems to work OK, my question is since I followed the perfect server install, it has me disable mail, I was wondering if there was a way to mail the logs to me using what ever the server is configured with (postfix,dovecot,???)?

Thanks
Reply With Quote
  #133  
Old 24th August 2011, 23:12
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Code:
            # If it's not the first day of the month we make incremental backup
            if [ ! -e $tmpdir/full-backup$XX.lck ] ; then
                log "Starting daily backup for: $YX"
                NEWER="--newer $FDATE"
                $TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
                log "Daily backup for $YX done."
            else
            log "Lock file for $YX full backup exists!"
            fi
Hi, I am running backup at 1 35 each night, so today, it will run at 25/08/2011 01:35 AM.

NEWER="--newer $FDATE"

with that line, it will check if a file is newer than 25/08/2011, and, if a file was created/modified between 24/08/2011 1:35 and 24/08/2011 23:59, it will be ignored.

I think that the solution should be to work with yesterday date and not with the actual date. It will copy twice files created between 25/08/2011 00:00 and 25/08/2011 1:35 (at next backup), but right now, incremental backup is doing almost nothing.

Is this correct or I forget to check something?

I will try this night with line value:
NEWER="--newer `date --date=yesterday +%F`"

instead of:
NEWER="--newer $FDATE"

But, I don't know is --newer checks only creation date or modified date too... we will see this night :P

Best regards

Edit: working ok, right now inc backup working, I think that this should be a modification for the scripts (or a custom value to modify depending on script running time.

Last edited by erosbk; 25th August 2011 at 12:38.
Reply With Quote
  #134  
Old 12th September 2011, 20:14
go0ogl3 go0ogl3 is offline
Member
 
Join Date: Jul 2008
Posts: 73
Thanks: 5
Thanked 39 Times in 6 Posts
Default

Quote:
Originally Posted by flyingscubakev View Post
Just a quick question from a relative newbie.
I've followed the perfect server - ispconfig3 on opesuse and everything is working, I've now loaded this backup script and it seems to work OK, my question is since I followed the perfect server install, it has me disable mail, I was wondering if there was a way to mail the logs to me using what ever the server is configured with (postfix,dovecot,???)?

Thanks
If you have a smtp mail account you can configure one of the multiple mail programs (mail, mutt, etc.) to send mail at the end of the script using that account.
__________________
__________________
go0ogl3
--
Keep your work safe. Better backup than sorry! Download the latest version of the backup and restore script. See more info here.
Reply With Quote
  #135  
Old 12th September 2011, 21:21
go0ogl3 go0ogl3 is offline
Member
 
Join Date: Jul 2008
Posts: 73
Thanks: 5
Thanked 39 Times in 6 Posts
Default

Quote:
Originally Posted by il.manuel View Post
hi google and thx for your great job!

i have two problem related on server rather than script.

my scenario: a VPS and a ftp backup server on remote machine with a 10GB quota on it.

i have mounted the remote ftp server using curlftpfs (http://curlftpfs.sourceforge.net/) and everything seems to be ok.

first problem: df show me the entire disk space on remote server (7,5TB !) and not my quota (10GB). max percentage calculation have no way to run well and the ftp space will end without a check possibility.

second problem: i thgink that curlftpfs not support append to file ('>>'). i got a lot of this error

Code:
./backupres: line 236: /backup/<<HOSTNAME>>/log/backup.log: Operation not supported
when he try to update logfile.

I think for an alternative, but i can't find a solution (manually deleting old backups!!!)

third problem

he not send me an email at the end of script. /tmp/tmpbck/maildata exist and he contains every log entry.

Hello,

the <<HOSTNAME>> is changed by you? If not that's the problem.

curlftpfs does not support append to the files. The solution is to move all the appended files away (log file, there is another?)
The alternative for the free space function is to get the free space with another method (repquota?) and use that value in the script...

The mail problem:
You need to have some mail server running (sendmail, postfix, dovecot, etc.) on your linux host to send mail with the back-res script...
I'll try to post a little script to send mail using SMTP AUTH and your ISP mail server (or another) and I'll try to modify the back-res script later this month.
__________________
__________________
go0ogl3
--
Keep your work safe. Better backup than sorry! Download the latest version of the backup and restore script. See more info here.
Reply With Quote
  #136  
Old 12th September 2011, 21:46
go0ogl3 go0ogl3 is offline
Member
 
Join Date: Jul 2008
Posts: 73
Thanks: 5
Thanked 39 Times in 6 Posts
Default

Quote:
Originally Posted by erosbk View Post
Code:
            # If it's not the first day of the month we make incremental backup
            if [ ! -e $tmpdir/full-backup$XX.lck ] ; then
                log "Starting daily backup for: $YX"
                NEWER="--newer $FDATE"
                $TAR $NEWER $ARG $BACKUPDIR/$MDATE/i$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded
                log "Daily backup for $YX done."
            else
            log "Lock file for $YX full backup exists!"
            fi
Hi, I am running backup at 1 35 each night, so today, it will run at 25/08/2011 01:35 AM.

NEWER="--newer $FDATE"

with that line, it will check if a file is newer than 25/08/2011, and, if a file was created/modified between 24/08/2011 1:35 and 24/08/2011 23:59, it will be ignored.

I think that the solution should be to work with yesterday date and not with the actual date. It will copy twice files created between 25/08/2011 00:00 and 25/08/2011 1:35 (at next backup), but right now, incremental backup is doing almost nothing.

Is this correct or I forget to check something?

I will try this night with line value:
NEWER="--newer `date --date=yesterday +%F`"

instead of:
NEWER="--newer $FDATE"

But, I don't know is --newer checks only creation date or modified date too... we will see this night :P

Best regards

Edit: working ok, right now inc backup working, I think that this should be a modification for the scripts (or a custom value to modify depending on script running time.

Hello,

From the tar manual seen on http://www.gnu.org/software/tar/manual/tar.html#SEC114:

Quote:
‘--newer=date’
‘-N date’
Only store files newer than date.

Acts on files only if their data modification or status change times are later than date. Use in conjunction with any operation.
The FDATE from the script is the actual date like
Code:
2011-09-12
which means 2011-09-12 time 00:00
In my tests I run the script at 00:01 on 8 september for example (first minute of the 8 day) and the script archives some files modified on
Code:
Sep  7 23:25
and on
Code:
Sep  8 00:00
so I can say it's doing his job....

How you use the script the backed up files are a day older, so you'll end up archiving more files on the incremental backup.

Use a simple test:
create a file in cron (touch) just before the actual run of the script... or use the tar command from the command line with the required arguments.
__________________
__________________
go0ogl3
--
Keep your work safe. Better backup than sorry! Download the latest version of the backup and restore script. See more info here.
Reply With Quote
  #137  
Old 12th September 2011, 22:01
go0ogl3 go0ogl3 is offline
Member
 
Join Date: Jul 2008
Posts: 73
Thanks: 5
Thanked 39 Times in 6 Posts
Default

Quote:
Originally Posted by fordwrench View Post
I have a server that has debian lenny and ispconfig via the perfect server howto.

I am trying to migrate to a new server I have created with debian squeeze viat the perfect server howto.

I make my backup and when I try to restore I get the following:



oot@srv1:~/scripts# sh /root/scripts/backres.sh db all 2011-07-16
date: invalid date `07/'
/root/scripts/backres.sh: line 448: [: : integer expression expected

You want to restore db all to date 2011-07-16.

Please input "yes" if the above is ok with you and press [ENTER]: yes

Restoring all mysql databases from date 2011-07-16 to local server:
touch: cannot touch `/tmp/tmpbck/datestart': No such file or directory
touch: cannot touch `/tmp/tmpbck/dateend': No such file or directory
mysql: unknown option '-x'
mysql: unknown option '-x'
mysql: unknown option '-x'
mysql: unknown option '-x'
mysql: unknown option '-x'
mysql: unknown option '-x'
mysql: unknown option '-x'
mysql: unknown option '-x'
mysql: unknown option '-x'
mysql: unknown option '-x'
All restore jobs done!
Database all restored to date 2011-07-16!
/root/scripts/backres.sh: line 573: /tmp/tmpbck/maildata: No such file or directory


Can you please give me some insight as to what the problem might be.
Sorry for seen your post so late, but I'll respond maybe others will need to restore too.

You can restore all of your files unpacking all the full_ archives, then all the incremental archives from that month OVER the full_ archives until the date needed. This way you'll end up with all your files, including the ones deleted from one incremental backup to another. You can use find --newer to delete unwanted files.

Hope this helps others too.

In your case it's possible that you have a problem with the date command and your /tmp dir is not writable or something.

The way I do the restore is using an empty partition or drive or a loop device mounted in a running system, I'm adding the needed /proc and others then make it bootable (grub, lilo, etc.), then remove the drive and run the restored system.
__________________
__________________
go0ogl3
--
Keep your work safe. Better backup than sorry! Download the latest version of the backup and restore script. See more info here.
Reply With Quote
  #138  
Old 12th September 2011, 22:08
go0ogl3 go0ogl3 is offline
Member
 
Join Date: Jul 2008
Posts: 73
Thanks: 5
Thanked 39 Times in 6 Posts
Smile Mail function / script for non-mail systems

Hello

here is a simple code for a mail function or script for the systems with mail server disabled:

Code:
#!/bin/bash
#
mailuser=`echo -n $email| base64`
mailpass=`echo -n $password| base64`

maildata=`$tmpdir/maildata`

nc mail.server.com 25 << EOF
EHLO test
auth login
$mailuser
$mailpass
mail from: $email
rcpt to: $email
DATA
Subject: test mail telnet

$maildata
.
quit
EOF
I'll try to modify the back-res script to add an option for this function. The code works only for mail servers which support smtp auth login but I hope will be enough...
__________________
__________________
go0ogl3
--
Keep your work safe. Better backup than sorry! Download the latest version of the backup and restore script. See more info here.
Reply With Quote
  #139  
Old 6th October 2011, 16:51
dar_ek dar_ek is offline
Junior Member
 
Join Date: Oct 2010
Posts: 17
Thanks: 1
Thanked 3 Times in 2 Posts
Default smal FIX for NFS disk usage problem

I noticed that if you store backup on a NFS disks when DELETE old backups NOT WORKS

I have standard maxp="85" but disks was full after some days.
Problem is in "function check_space"
Line:
Code:
  pfs=`df -h $BACKUPDIR | awk 'NR==2{print $5}' | cut -d% -f 1`
In common disks it works cause all infos from df are in one line but in NFS mounted disks is IN TWO LINES, look:

Code:
# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              14G  1,7G   12G  13% /
10.246.120.212:/home/BACKUP/POCZTA
                       65G   65G     0 100% /home/BACKUP
in this case you can user "P" switch in df, that tell df to dont do this (2 lines output for one disk).
And fix will look that:

Code:
 pfs=`df -Ph $BACKUPDIR | awk 'NR==2{print $5}' | cut -d% -f 1`

please check that "df -Ph" will be works on other linux distros, and if its woks to change that.

regards
Reply With Quote
The Following 2 Users Say Thank You to dar_ek For This Useful Post:
falko (7th October 2011), go0ogl3 (12th October 2011)
  #140  
Old 12th October 2011, 22:00
go0ogl3 go0ogl3 is offline
Member
 
Join Date: Jul 2008
Posts: 73
Thanks: 5
Thanked 39 Times in 6 Posts
 
Thumbs up Thanks for pointing this up!

Quote:
Originally Posted by dar_ek View Post
I noticed that if you store backup on a NFS disks when DELETE old backups NOT WORKS

I have standard maxp="85" but disks was full after some days.
Problem is in "function check_space"
Line:
Code:
  pfs=`df -h $BACKUPDIR | awk 'NR==2{print $5}' | cut -d% -f 1`
In common disks it works cause all infos from df are in one line but in NFS mounted disks is IN TWO LINES, look:

Code:
# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              14G  1,7G   12G  13% /
10.246.120.212:/home/BACKUP/POCZTA
                       65G   65G     0 100% /home/BACKUP
in this case you can user "P" switch in df, that tell df to dont do this (2 lines output for one disk).
And fix will look that:

Code:
 pfs=`df -Ph $BACKUPDIR | awk 'NR==2{print $5}' | cut -d% -f 1`

please check that "df -Ph" will be works on other linux distros, and if its woks to change that.

regards
Thank you for help, this is because your NFS path is long but is a small fix wich I'll include in the next version, sometime in december, if my timing is ok.
__________________
__________________
go0ogl3
--
Keep your work safe. Better backup than sorry! Download the latest version of the backup and restore script. See more info here.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
freebsd 7, samba 3, domain controller alexdimarco Suggest HOWTO 6 5th November 2010 17:54
how to restore Ispconfig from backup db na0lb Installation/Configuration 4 24th September 2008 09:51
Restore backup ISPConfig leeeonardo General 4 18th April 2007 21:21
Restore ispconfig complete backup statiic General 1 15th January 2007 11:32
Backup restore on ispconfig misterm Installation/Configuration 0 19th December 2005 00:32


All times are GMT +2. The time now is 18:42.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.