Creating Encrypted FTP Backups With duplicity And duply On Debian Squeeze - Page 2

Want to support HowtoForge? Become a subscriber!
 
Submitted by falko (Contact Author) (Forums) on Thu, 2012-07-05 18:07. ::

We can now create three other files, although that is totally optional:

  • /root/.duply/exampleuser/exclude: contains a list of directories to be excluded from the backup (one directory per line);
  • /root/.duply/exampleuser/pre: contains command(s) to be executed prior to the backup (e.g. create a MySQL database dump);
  • /root/.duply/exampleuser/post: contains command(s) to be executed after the backup.

Here's a sample /root/.duply/exampleuser/exclude and /root/.duply/exampleuser/pre file (the syntax of /root/.duply/exampleuser/post is the same as in /root/.duply/exampleuser/pre):

vi /root/.duply/exampleuser/exclude

/home/exampleuser/news
/home/exampleuser/messages
/home/exampleuser/tmp

chmod 600 /root/.duply/exampleuser/exclude

vi /root/.duply/exampleuser/pre

/usr/bin/mysqldump --all-databases -u root -pyourrootsqlpassword > /home/exampleuser/db.sql

(It should be noted that the database gets locked during the creation of SQL dump; this might not be an issue for a small web site, but can be a problem for high-traffic web sites - your visitors won't be able to access database-driven pages during mysqldump. Here's a link to an interruption-free MySQL backup method: How To Back Up MySQL Databases Without Interrupting MySQL)

/root/.duply/exampleuser/pre must be executable (as well as /root/.duply/exampleuser/post if you choose to create one):

chmod 700 /root/.duply/exampleuser/pre

As I said before, you only need /root/.duply/exampleuser/conf; the other files are optional.

Now duply is ready to be used; to create our backup, we simply run

duply exampleuser backup

If all goes well, you won't be prompted for a password:

root@server1:~# duply exampleuser backup
Start duply v1.5.7, time is 2012-07-03 21:02:33.
Using profile '/root/.duply/exampleuser'.
Using installed duplicity version 0.6.08b, python 2.6.6, gpg 1.4.10 (Home: ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan', bash '4.1.5(1)-release (x86_64-pc-linux-gnu)'.
Autoset found secret key of first GPG_KEY entry '7C6E958B' for signing.
Test - Encrypt to 7C6E958B & Sign with 7C6E958B (OK)
Test - Decrypt (OK)
Test - Compare (OK)
Cleanup - Delete '/tmp/duply.4161.1341342154_*'(OK)

--- Start running command PRE at 21:02:34.459 ---
Running '/root/.duply/exampleuser/pre' - OK
--- Finished state OK at 21:02:34.535 - Runtime 00:00:00.075 ---

--- Start running command BKP at 21:02:34.599 ---
NcFTP version is 3.2.4
Reading globbing filelist /root/.duply/exampleuser/exclude
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Tue Jul 3 20:47:50 2012
--------------[ Backup Statistics ]--------------
StartTime 1341342156.07 (Tue Jul 3 21:02:36 2012)
EndTime 1341342156.13 (Tue Jul 3 21:02:36 2012)
ElapsedTime 0.06 (0.06 seconds)
SourceFiles 50
SourceFileSize 13490043 (12.9 MB)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 0
RawDeltaSize 0 (0 bytes)
TotalDestinationSizeChange 716 (716 bytes)
Errors 0
-------------------------------------------------

--- Finished state OK at 21:02:37.483 - Runtime 00:00:02.884 ---

--- Start running command POST at 21:02:37.556 ---
Skipping n/a script '/root/.duply/exampleuser/post'.
--- Finished state OK at 21:02:37.636 - Runtime 00:00:00.080 ---
root@server1:~#

To automate the backups, we can create a cron job (I create two cron jobs, one that runs daily and creates incremental backups, and one that runs once a month, creates a full backup and deletes old files):

crontab -e

# run the (incremental) backup each night at 03:23h
23 3 * * * /usr/local/bin/duply exampleuser backup

# do a full backup once per month & delete old backups
47 4 1 * * /usr/local/bin/duply exampleuser full && /usr/local/bin/duply exampleuser purge --force

 

6 Restoring A Backup With duply

Of course, you can use duply also to restore a backup (again, you won't be asked for a password). The syntax is very easy:

duply exampleuser restore /home/exampleuser

You can take a look at the duply help to learn what other options you have:

duply -h

root@server1:~# duply -h
VERSION:
  duply version 1.5.7
  (http://duply.net)

DESCRIPTION:
  Duply deals as a wrapper for the mighty duplicity magic.
  It simplifies running duplicity with cron or on command line by:

    - keeping recurring settings in profiles per backup job
    - enabling batch operations eg. backup_verify_purge
    - executing pre/post scripts for every command
    - precondition checking for flawless duplicity operation

  For each backup job one configuration profile must be created.
  The profile folder will be stored under '~/.duply/<profile>'
  (where ~ is the current users home directory).
  Hint:
   If the folder '/etc/duply' exists, the profiles for the super
   user root will be searched & created there.

USAGE:
  first time usage (profile creation):
    duply <profile> create

  general usage in single or batch mode (see EXAMPLES):
    duply <profile> <command>[_<command>_...] [<options> ...]

  Non duply options are passed on to duplicity (see OPTIONS).
  All conf parameters can also be defined in the environment instead.

PROFILE:
  Indicated by a path or a profile name (<profile>), which is resolved
  to '~/.duply/<profile>' (~ expands to environment variable $HOME).

  Superuser root can place profiles under '/etc/duply'. Simply create
  the folder manually before running duply as superuser.
  Note:
    Already existing profiles in root's profile folder will cease to work
    unless there are moved to the new location manually.

  example 1:   duply humbug backup

  Alternatively a _path_ might be used e.g. useful for quick testing,
  restoring or exotic locations. Shell expansion should work as usual.
  Hint:
    The path must contain at least one path separator '/',
    e.g. './test' instead of only 'test'.

  example 2:   duply ~/.duply/humbug backup

COMMANDS:
  usage      get usage help text

  create     creates a configuration profile
  backup     backup with pre/post script execution (batch: pre_bkp_post),
              full (if full_if_older matches or no earlier backup is found)
              incremental (in all other cases)
  pre/post   execute '<profile>/pre', '<profile>/post' scripts
  bkp        as above but without executing pre/post scripts
  full       force full backup
  incr       force incremental backup
  list [<age>]
             list all files in backup (as it was at <age>, default: now)
  status     prints backup sets and chains currently in repository
  verify     list files changed since latest backup
  restore <target_path> [<age>]
             restore the complete backup to <target_path> [as it was at <age>]
  fetch <src_path> <target_path> [<age>]
             fetch single file/folder from backup [as it was at <age>]
  purge [<max_age>] [--force]
             list outdated backup files (older than $MAX_AGE)
              [use --force to actually delete these files]
  purge-full [<max_full_backups>] [--force]
             list outdated backup files ($MAX_FULL_BACKUPS being the number of
             full backups and associated incrementals to keep, counting in
             reverse chronological order)
              [use --force to actually delete these files]
  cleanup [--force]
             list broken backup chain files archives (e.g. after unfinished run)
              [use --force to actually delete these files]

  changelog  print changelog / todo list
  txt2man    feature for package maintainers - create a manpage based on the
             usage output. download txt2man from http://mvertes.free.fr/, put
             it in the PATH and run 'duply txt2man' to create a man page.

OPTIONS:
  --force    passed to duplicity (see commands: purge, purge-full, cleanup)
  --preview  do nothing but print out generated duplicity command lines
  --disable-encryption
             disable encryption, overrides profile settings

PRE/POST SCRIPTS:
  All internal duply variables will be readable in the scripts.
  Some of interest might be

    CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>,
    GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_<PREV|NEXT>

  The CMD_* variables were introduced to allow different actions according to
  the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post'
  will call the pre script two times, with CMD_NEXT variable set to 'bkp'
  on the first and to 'verify' on the second run.

EXAMPLES:
  create profile 'humbug':
    duply humbug create (now edit the resulting conf file)
  backup 'humbug' now:
    duply humbug backup
  list available backup sets of profile 'humbug':
    duply humbug status
  list and delete obsolete backup archives of 'humbug':
    duply humbug purge --force
  restore latest backup of 'humbug' to /mnt/restore:
    duply humbug restore /mnt/restore
  restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:
    duply humbug fetch etc/passwd /root/pw 4D
    (see "duplicity manpage", section TIME FORMATS)
  a one line batch job on 'humbug' for cron execution:
    duply humbug backup_verify_purge --force

FILES:
  in profile folder '~/.duply/<profile>' or '/etc/duply'
  conf             profile configuration file
  pre,post         pre/post scripts (see above for details)
  gpgkey.*.asc     exported GPG key files
  exclude          a globbing list of included or excluded files/folders
                   (see "duplicity manpage", section FILE SELECTION)

IMPORTANT:
  Copy the _whole_ profile folder after the first backup to a safe place.
  It contains everything needed to restore your backups. You will need
  it if you have to restore the backup from another system (e.g. after a
  system crash). Keep access to these files restricted as they contain
  _all_ informations (gpg data, ftp data) to access and modify your backups.

  Repeat this step after _all_ configuration changes. Some configuration
  options are crucial for restoration.

SEE ALSO:
  duplicity man page:
    duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
root@server1:~#

 

7 Links


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.