Creating Encrypted FTP Backups With duplicity And duply On Debian Squeeze - Page 2
On this page
We can now create three other files, although that is totally optional:
- /root/.duply/exampleuser/exclude: contains a list of directories to be excluded from the backup (one directory per line);
- /root/.duply/exampleuser/pre: contains command(s) to be executed prior to the backup (e.g. create a MySQL database dump);
- /root/.duply/exampleuser/post: contains command(s) to be executed after the backup.
Here's a sample /root/.duply/exampleuser/exclude and /root/.duply/exampleuser/pre file (the syntax of /root/.duply/exampleuser/post is the same as in /root/.duply/exampleuser/pre):
vi /root/.duply/exampleuser/exclude
/home/exampleuser/news /home/exampleuser/messages /home/exampleuser/tmp |
chmod 600 /root/.duply/exampleuser/exclude
vi /root/.duply/exampleuser/pre
/usr/bin/mysqldump --all-databases -u root -pyourrootsqlpassword > /home/exampleuser/db.sql |
(It should be noted that the database gets locked during the creation of SQL dump; this might not be an issue for a small web site, but can be a problem for high-traffic web sites - your visitors won't be able to access database-driven pages during mysqldump. Here's a link to an interruption-free MySQL backup method: How To Back Up MySQL Databases Without Interrupting MySQL)
/root/.duply/exampleuser/pre must be executable (as well as /root/.duply/exampleuser/post if you choose to create one):
chmod 700 /root/.duply/exampleuser/pre
As I said before, you only need /root/.duply/exampleuser/conf; the other files are optional.
Now duply is ready to be used; to create our backup, we simply run
duply exampleuser backup
If all goes well, you won't be prompted for a password:
root@server1:~# duply exampleuser backup
Start duply v1.5.7, time is 2012-07-03 21:02:33.
Using profile '/root/.duply/exampleuser'.
Using installed duplicity version 0.6.08b, python 2.6.6, gpg 1.4.10 (Home: ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan', bash '4.1.5(1)-release (x86_64-pc-linux-gnu)'.
Autoset found secret key of first GPG_KEY entry '7C6E958B' for signing.
Test - Encrypt to 7C6E958B & Sign with 7C6E958B (OK)
Test - Decrypt (OK)
Test - Compare (OK)
Cleanup - Delete '/tmp/duply.4161.1341342154_*'(OK)
--- Start running command PRE at 21:02:34.459 ---
Running '/root/.duply/exampleuser/pre' - OK
--- Finished state OK at 21:02:34.535 - Runtime 00:00:00.075 ---
--- Start running command BKP at 21:02:34.599 ---
NcFTP version is 3.2.4
Reading globbing filelist /root/.duply/exampleuser/exclude
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Tue Jul 3 20:47:50 2012
--------------[ Backup Statistics ]--------------
StartTime 1341342156.07 (Tue Jul 3 21:02:36 2012)
EndTime 1341342156.13 (Tue Jul 3 21:02:36 2012)
ElapsedTime 0.06 (0.06 seconds)
SourceFiles 50
SourceFileSize 13490043 (12.9 MB)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 0
RawDeltaSize 0 (0 bytes)
TotalDestinationSizeChange 716 (716 bytes)
Errors 0
-------------------------------------------------
--- Finished state OK at 21:02:37.483 - Runtime 00:00:02.884 ---
--- Start running command POST at 21:02:37.556 ---
Skipping n/a script '/root/.duply/exampleuser/post'.
--- Finished state OK at 21:02:37.636 - Runtime 00:00:00.080 ---
root@server1:~#
To automate the backups, we can create a cron job (I create two cron jobs, one that runs daily and creates incremental backups, and one that runs once a month, creates a full backup and deletes old files):
crontab -e
# run the (incremental) backup each night at 03:23h 23 3 * * * /usr/local/bin/duply exampleuser backup # do a full backup once per month & delete old backups 47 4 1 * * /usr/local/bin/duply exampleuser full && /usr/local/bin/duply exampleuser purge --force |
6 Restoring A Backup With duply
Of course, you can use duply also to restore a backup (again, you won't be asked for a password). The syntax is very easy:
duply exampleuser restore /home/exampleuser
You can take a look at the duply help to learn what other options you have:
duply -h
root@server1:~# duply -h
VERSION:
duply version 1.5.7
(http://duply.net)
DESCRIPTION:
Duply deals as a wrapper for the mighty duplicity magic.
It simplifies running duplicity with cron or on command line by:
- keeping recurring settings in profiles per backup job
- enabling batch operations eg. backup_verify_purge
- executing pre/post scripts for every command
- precondition checking for flawless duplicity operation
For each backup job one configuration profile must be created.
The profile folder will be stored under '~/.duply/<profile>'
(where ~ is the current users home directory).
Hint:
If the folder '/etc/duply' exists, the profiles for the super
user root will be searched & created there.
USAGE:
first time usage (profile creation):
duply <profile> create
general usage in single or batch mode (see EXAMPLES):
duply <profile> <command>[_<command>_...] [<options> ...]
Non duply options are passed on to duplicity (see OPTIONS).
All conf parameters can also be defined in the environment instead.
PROFILE:
Indicated by a path or a profile name (<profile>), which is resolved
to '~/.duply/<profile>' (~ expands to environment variable $HOME).
Superuser root can place profiles under '/etc/duply'. Simply create
the folder manually before running duply as superuser.
Note:
Already existing profiles in root's profile folder will cease to work
unless there are moved to the new location manually.
example 1: duply humbug backup
Alternatively a _path_ might be used e.g. useful for quick testing,
restoring or exotic locations. Shell expansion should work as usual.
Hint:
The path must contain at least one path separator '/',
e.g. './test' instead of only 'test'.
example 2: duply ~/.duply/humbug backup
COMMANDS:
usage get usage help text
create creates a configuration profile
backup backup with pre/post script execution (batch: pre_bkp_post),
full (if full_if_older matches or no earlier backup is found)
incremental (in all other cases)
pre/post execute '<profile>/pre', '<profile>/post' scripts
bkp as above but without executing pre/post scripts
full force full backup
incr force incremental backup
list [<age>]
list all files in backup (as it was at <age>, default: now)
status prints backup sets and chains currently in repository
verify list files changed since latest backup
restore <target_path> [<age>]
restore the complete backup to <target_path> [as it was at <age>]
fetch <src_path> <target_path> [<age>]
fetch single file/folder from backup [as it was at <age>]
purge [<max_age>] [--force]
list outdated backup files (older than $MAX_AGE)
[use --force to actually delete these files]
purge-full [<max_full_backups>] [--force]
list outdated backup files ($MAX_FULL_BACKUPS being the number of
full backups and associated incrementals to keep, counting in
reverse chronological order)
[use --force to actually delete these files]
cleanup [--force]
list broken backup chain files archives (e.g. after unfinished run)
[use --force to actually delete these files]
changelog print changelog / todo list
txt2man feature for package maintainers - create a manpage based on the
usage output. download txt2man from http://mvertes.free.fr/, put
it in the PATH and run 'duply txt2man' to create a man page.
OPTIONS:
--force passed to duplicity (see commands: purge, purge-full, cleanup)
--preview do nothing but print out generated duplicity command lines
--disable-encryption
disable encryption, overrides profile settings
PRE/POST SCRIPTS:
All internal duply variables will be readable in the scripts.
Some of interest might be
CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>,
GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_<PREV|NEXT>
The CMD_* variables were introduced to allow different actions according to
the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post'
will call the pre script two times, with CMD_NEXT variable set to 'bkp'
on the first and to 'verify' on the second run.
EXAMPLES:
create profile 'humbug':
duply humbug create (now edit the resulting conf file)
backup 'humbug' now:
duply humbug backup
list available backup sets of profile 'humbug':
duply humbug status
list and delete obsolete backup archives of 'humbug':
duply humbug purge --force
restore latest backup of 'humbug' to /mnt/restore:
duply humbug restore /mnt/restore
restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:
duply humbug fetch etc/passwd /root/pw 4D
(see "duplicity manpage", section TIME FORMATS)
a one line batch job on 'humbug' for cron execution:
duply humbug backup_verify_purge --force
FILES:
in profile folder '~/.duply/<profile>' or '/etc/duply'
conf profile configuration file
pre,post pre/post scripts (see above for details)
gpgkey.*.asc exported GPG key files
exclude a globbing list of included or excluded files/folders
(see "duplicity manpage", section FILE SELECTION)
IMPORTANT:
Copy the _whole_ profile folder after the first backup to a safe place.
It contains everything needed to restore your backups. You will need
it if you have to restore the backup from another system (e.g. after a
system crash). Keep access to these files restricted as they contain
_all_ informations (gpg data, ftp data) to access and modify your backups.
Repeat this step after _all_ configuration changes. Some configuration
options are crucial for restoration.
SEE ALSO:
duplicity man page:
duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
root@server1:~#
7 Links
- duplicity: http://duplicity.nongnu.org/
- duply: http://duply.net/
- Debian: http://www.debian.org/
