Maintaining Remote Web Sites With sitecopy (Debian Squeeze/Ubuntu 11.10)

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Follow me on Twitter
Last edited 04/17/2012

sitecopy is a tool for copying locally stored web sites to a remote web server (using FTP or WebDAV). It helps you to keep the remote site synchronized with your local copy by uploading modified local files and deleting remote files that have been deleted on the local computer. This tutorial shows how you can manage your remote web site from your local Debian Squeeze/Ubuntu 11.10 desktop with sitecopy.

I do not issue any guarantee that this will work for you!

1 Preliminary Note

I'm using the username falko on my local Debian Squeeze/Ubuntu 11.10 desktop (I'm logged in on my local Linux desktop as that user - please don't log in as root). The files for the remote web site example.com are stored in the directory /home/falko/sites/example.com/ on the local computer. The remote document root is /var/www/example.com/web/.

You can use sitecopy with FTP and WebDAV, so you should either have an FTP or a WebDAV account on the remote server. I'm using the FTP/WebDAV username defaultfalko and the password howtoforge here.

2 Installing sitecopy

sitecopy can be installed on the local desktop as follows (we need root privileges, therefore we use sudo):

sudo apt-get install sitecopy

You should now take a look at sitecopy's man page to familiarize yourself with its options:

man sitecopy

3 Configuring sitecopy

Go to your home directory on the local desktop...

cd ~

... and create the directory .sitecopy with permissions of 700 (sitecopy uses that directory to store file details):

mkdir -m 700 .sitecopy

Next create the sitecopy configuration file .sitecopyrc:

touch .sitecopyrc
chmod 600 .sitecopyrc

Open the file...

vi .sitecopyrc

... and fill in the configuration for the example.com site. Here are two examples, one for FTP...

site example.com
  server example.com
  username defaultfalko
  password howtoforge
  local /home/administrator/sites/example.com/
  remote ~/web/
  exclude *.bak
  exclude *~

... and one for WebDAV:

site example.com
  server example.com
  protocol webdav
  username defaultfalko
  password howtoforge
  local /home/administrator/sites/example.com/
  remote /var/www/example.com/web/
  exclude *.bak
  exclude *~

(You can define a stanza for each web site you want to manage with sitecopy.)

The site directive must be followed by a name for the web site - you can freely choose one, e.g. example.com or mysite. This name will be used later on in the sitecopy commands. The following configuration options that belong to that site must be indented!

Most of the following configuration options are self-explaining. The default protocol is FTP; if you want to use WebDAV, please specify protocol webdav. The local ditective contains the local path of the web site copy, remote contains the path of the web site on the remote server - it can be absolute or relative. If your user is chrooted (as is normally the case with FTP users), you should use a relative path (such as ~/ or ~/web). Otherwise use an absolute path.

The exclude lines are optional, they are here just to demonstrate how you can exclude files from being maintained by sitecopy.

You can find out more about sitecopy configuration on its man page:

man sitecopy

4 First Usage

Before you use sitecopy for the first time, you have to decide which of the following three scenarios matches your situation:

1. Existing remote site and local copy, both in sync.
2. Existing remote site, no local copy.
3. New remote site, existing local copy.

4.1 Existing Remote Site And local Copy, Both In Sync

If both the remote site and the local copy exist and are in sync, run

sitecopy --catchup example.com

to make sitecopy think the local site is exactly the same as the remote copy. Replace example.com with the name of the site you use in the .sitecopyrc file.

falko@falko-desktop:~$ sitecopy --catchup example.com
sitecopy: Catching up site `example.com' (on example.com in ~/web/)
sitecopy: All the files and and directories are marked as updated remotely.
falko@falko-desktop:~$

4.2 Existing Remote Site, No Local Copy

If you have no local copy of the existing remote web site, run

sitecopy --fetch example.com

first so that sitecopy fetches the list of files from the remote server (replace example.com with the name of the site you use in the .sitecopyrc file):

falko@falko-desktop:~$ sitecopy --fetch example.com
sitecopy: Fetching site `example.com' (on example.com in ~/web/)
File: data/index.html - size 5
File: error/503.html - size 1906
File: error/502.html - size 1881
File: error/500.html - size 1851
File: error/405.html - size 1810
File: error/404.html - size 1806
File: error/403.html - size 1809
File: error/401.html - size 1806
File: error/400.html - size 1792
File: stats/.htaccess - size 116
File: robots.txt - size 24
File: index.html - size 1861
File: favicon.ico - size 7358
File: .htaccess - size 26
Directory: data/
Directory: error/
Directory: stats/
sitecopy: Fetch completed successfully.
falko@falko-desktop:~$

Then run

sitecopy --synch example.com

to update the local site from the remote copy.

falko@falko-desktop:~$ sitecopy --synch example.com
sitecopy: Synchronizing site `example.com' (on example.com in ~/web/)
Creating data/: done.
Creating error/: done.
Creating stats/: done.
Downloading data/index.html: [.] done.
Downloading error/503.html: [.] done.
Downloading error/502.html: [.] done.
Downloading error/500.html: [.] done.
Downloading error/405.html: [.] done.
Downloading error/404.html: [.] done.
Downloading error/403.html: [.] done.
Downloading error/401.html: [.] done.
Downloading error/400.html: [.] done.
Downloading stats/.htaccess: [.] done.
Downloading robots.txt: [.] done.
Downloading index.html: [.] done.
Downloading favicon.ico: [.] done.
Downloading .htaccess: [.] done.
sitecopy: Synchronize completed successfully.
falko@falko-desktop:~$

4.3 New Remote Site, Existing Local Copy

If the local copy exists, but you have an empty remote site, run

sitecopy --init example.com

first to intialize the site. Replace example.com with the name of the site you use in the .sitecopyrc file.

falko@falko-desktop:~$ sitecopy --init example.com
sitecopy: Initializing site `example.com' (on example.com in ~/web/)
sitecopy: All the files and directories are marked as NOT updated remotely.
falko@falko-desktop:~$

Then run

sitecopy --update example.com

to upload the local copy to the remote site:

falko@falko-desktop:~$ sitecopy --update example.com
sitecopy: Updating site `example.com' (on example.com in ~/web/)
Creating stats/: done.
Creating data/: done.
Creating error/: done.
Uploading stats/.htaccess: [.] done.
Uploading data/index.html: [.] done.
Uploading error/403.html: [.] done.
Uploading error/401.html: [.] done.
Uploading error/404.html: [.] done.
Uploading error/503.html: [.] done.
Uploading error/400.html: [.] done.
Uploading error/502.html: [.] done.
Uploading error/405.html: [.] done.
Uploading error/500.html: [.] done.
Uploading index.html: [.] done.
Uploading robots.txt: [.] done.
Uploading .htaccess: [.] done.
Uploading favicon.ico: [.] done.
sitecopy: Update completed successfully.
falko@falko-desktop:~$

5 Using sitecopy

Afterwards, sitecopy usage is really easy. You can work with your local copy and update, create, and delete files. A first, but optional step is to run

sitecopy example.com

to find out which files have changed locally (replace example.com with the name of the site you use in the .sitecopyrc file):

falko@falko-desktop:~$ sitecopy example.com
sitecopy: Showing changes to site `example.com' (on example.com in ~/web/)
* These items have been added since the last update:
info.php
sitecopy: The remote site needs updating (1 item to update).
falko@falko-desktop:~$

To synchronize your remote web site with your local copy (i.e. upload new and changed files to the remote server and delete files on the remote server that have been deleted locally), you simply run

sitecopy --update example.com

falko@falko-desktop:~$ sitecopy --update example.com
sitecopy: Updating site `example.com' (on example.com in ~/web/)
Uploading info.php: [.] done.
sitecopy: Update completed successfully.
falko@falko-desktop:~$

That's it! Have fun with sitecopy!

6 Links

• sitecopy: http://www.manyfish.co.uk/sitecopy/
• Debian: http://www.debian.org/
• Ubuntu: http://www.ubuntu.com/