Mirror Your Web Site With rsync On Fedora 10

Version 1.0
Author: Falko Timme
Last edited 01/26/2009

This tutorial shows how you can mirror your web site from your main web server to a backup server (both running Fedora 10) that can take over if the main server fails. We use the tool rsync for this, and we make it run through a cron job that checks every x minutes if there is something to update on the mirror. Thus your backup server should usually be up to date if it has to take over.

rsync updates only files that have changed, so you do not need to transfer 5 GB of data whenever you run rsync. It only mirrors new/changed files, and it can also delete files from the mirror that have been deleted on the main server. In addition to that it can preserve permissions and ownerships of mirrored files and directories; to preserve the ownerships, we need to run rsync as root which is what we do here. If permissions and/or ownerships change on the main server, rsync will also change them on the backup server.

In this tutorial we will tunnel rsync through SSH which is more secure; it also means you do not have to open another port in your firewall for rsync - it is enough if port 22 (SSH) is open. The problem is that SSH requires a password for logging in which is not good if you want to run rsync as a cron job. The need for a password requires human interaction which is not what we want.

But fortunately there is a solution: the use of public keys. We create a pair of keys (on our backup server server2.example.com), one of which is saved in a file on the remote system (server1.example.com). Afterwards we will not be prompted for a password anymore when we run rsync. This also includes cron jobs which is exactly what we want.

As you might have guessed already from what I have written so far, the concept is that we initiate the mirroring of server1.example.com directly from server2.example.com; server1.example.com does not have to do anything to get mirrored.

I will use the following setup here:

  • Main server (Fedora 10): server1.example.com (server1) - IP address: 192.168.0.100
  • Mirror/backup server (Fedora 10): server2.example.com (server2) - IP address: 192.168.0.101
  • The web site that is to be mirrored is in /var/www/html on server1.example.com.

rsync is for mirroring files and directories only; if you want to mirror your MySQL database, please take a look at these tutorials:

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

 

1 Install rsync

First we have to install rsync on both server1.example.com and server2.example.com:

server1/server2:

(We do this as root!)

yum install rsync

 

2 Create An Unprivileged User On server1.example.com

Now we create an unprivileged user called someuser on server1.example.com that will be used by rsync on server2.example.com to mirror the directory /var/www/html (of course, someuser must have read permissions on /var/www/html on server1.example.com).

server1:

(We do this as root!)

useradd -d /home/someuser -m -s /bin/bash someuser

This will create the user someuser with the home directory /home/someuser and the login shell /bin/bash (it is important that someuser has a valid login shell - something like /bin/false does not work!). Now give someuser a password:

passwd someuser

 

3 Test rsync

Next we test rsync on server2.example.com. As root we do this:

server2:

rsync -avz -e ssh someuser@server1.example.com:/var/www/html/ /var/www/html/

You should see something like this. Answer with yes:

[root@server2 ~]# rsync -avz -e ssh someuser@server1.example.com:/var/www/html/ /var/www/html/
The authenticity of host 'server1.example.com (192.168.0.100)' can't be established.
RSA key fingerprint is 96:1e:9d:39:25:40:9d:89:53:f6:71:8f:fd:79:89:18.
Are you sure you want to continue connecting (yes/no)?
<-- yes

Then enter someuser's password, and you should see that server1.example.com's /var/www/html directory is mirrored to /var/www/html on server2.example.com.

You can check that like this on both servers:

server1/server2:

ls -la /var/www/html

You should see that all files and directories have been mirrored to server2.example.com, and the files and directories should have the same permissions/ownerships as on server1.example.com.

 

4 Create The Keys On server2.example.com

Now we create the private/public key pair on server2.example.com:

server2:

(We do this as root!)

mkdir /root/rsync
ssh-keygen -t dsa -b 1024 -f /root/rsync/mirror-rsync-key

You will see something like this:

[root@server2 ~]# ssh-keygen -t dsa -b 1024 -f /root/rsync/mirror-rsync-key
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase)
<-- ENTER
Enter same passphrase again: <-- ENTER
Your identification has been saved in /root/rsync/mirror-rsync-key.
Your public key has been saved in /root/rsync/mirror-rsync-key.pub.
The key fingerprint is:
68:1e:9c:12:f1:f5:7f:53:d5:1d:d0:f2:dd:c2:88:f3 root@server2.example.com
The key's randomart image is:
+--[ DSA 1024]----+
|    .   .    .o.=|
|     o . .   . .+|
|    . .   .. oo +|
|     o o  o.. ooo|
|    . * S  o. o. |
|     + .    E. . |
|      .          |
|                 |
|                 |
+-----------------+
[root@server2 ~]#

It is important that you do not enter a passphrase otherwise the mirroring will not work without human interaction so simply hit enter!

Next, we copy our public key to server1.example.com:

server2:

(Still, we do this as root.)

scp /root/rsync/mirror-rsync-key.pub someuser@server1.example.com:/home/someuser/

The public key mirror-rsync-key.pub should now be available in /home/someuser on server1.example.com.

Share this page:

4 Comment(s)

Add comment

Comments

From: Ignacio Sánchez at: 2011-04-14 14:14:41

lsync is a good tool which rsync the two servers automatically. Really easy to config

From: Nelson at: 2010-01-13 10:29:53

Hi,

 know that this is not the place, where a person should ask for help, but I found something that probably a few others may find too, and it would be a shame if for a small thing this how-to could not produce the desired results.

Anyway, I found that (in my case) the authorized_keys file does not work as it should. After some trial and error, I discovered that using the parameter ' from="server2.example.com" ' result in the rsync command from server1 asking for the password for 'someuser'. If we take this parameter from the authorized_keys file, the rsync command works like a charm.

 I feel that this may lead to an unsecure system, since it may allow a rsync command from any other host, but since i am not sure, this would be a doubt i have.

 Thanks in advance, Nelson Ribeiro.

From: joseph at: 2011-01-20 22:59:50

I found out after several hours of redoing these steps that if you are ssh-ing into any of the computers, ie server1 or server2, you need to ssh into the fqdn to do the key generator and rsync-ing.  Other wise the keys will not match and you will need to put in the someuser's password.

 fyi:

Fedora13, clean install on 6 machines

with: http://www.howtoforge.com/mysql_database_replication

and: http://www.howtoforge.com/high-availability-load-balancer-haproxy-heartbeat-fedora8, but changed to keepalived instead of heartbeat.

From: Senthil at: 2012-02-27 13:38:45

Hi,

 Its work fine...But its asking password when ever connect to master.Also crontab can't be work by this reason.

 Please  any one help me.

 Thanks