Maintaining remote web sites with sitecopy on Ubuntu 16.04

sitecopy is a tool for copying locally stored web sites to a remote web server (using FTP or WebDAV). It helps you to keep the remote site synchronized with your local copy by uploading modified local files and deleting remote files that have been deleted on the local computer. This tutorial shows how you can manage your remote web site from your local Ubuntu 16.04 desktop with sitecopy.

 

1 Preliminary Note

I'm using the username till on my local Ubuntu desktop (I'm logged in on my local Linux desktop as that user - please don't log in as root). The files for the remote web site example.com are stored in the directory /home/till/sites/example.com/ on the local computer. The remote document root is /var/www/example.com/web/.

You can use sitecopy with FTP and WebDAV, so you should either have an FTP or a WebDAV account on the remote server. I'm using the FTP username defaulttill and the password howtoforge here.

The remote website that I use for these examples has been created with ISPConfig. But any site where you have FTP or WebDAV access to will work.

 

2 Installing sitecopy

Sitecopy can be installed on the local desktop as follows (we need root privileges, therefore we use sudo):

sudo apt-get -y install sitecopy

You should now take a look at sitecopy's man page to familiarize yourself with its options:

man sitecopy

 

3 Configuring sitecopy

Go to your home directory on the local desktop...

cd ~

... and create the directory .sitecopy with permissions of 700 (sitecopy uses that directory to store file details):

mkdir -m 700 .sitecopy

Next, create the sitecopy configuration file .sitecopyrc:

touch .sitecopyrc
chmod 600 .sitecopyrc

Open the file...

nano .sitecopyrc

... and fill in the configuration for the example.com site. Here are two examples, one for FTP...

site example.com
  server example.com
  username defaulttill
  password howtoforge
  local /home/till/sites/example.com/
  remote ~/web/
  exclude *.bak
  exclude *~

... and one for WebDAV:

site example.com
  server example.com
  protocol webdav
  username defaulttill
  password howtoforge
  local /home/till/sites/example.com/
  remote /var/www/example.com/web/
  exclude *.bak
  exclude *~

(You can define a stanza for each web site you want to manage with sitecopy.)

The site directive must be followed by a name for the web site - you can freely choose one, e.g. example.com or mysite. This name will be used later on in the sitecopy commands. The following configuration options that belong to that site must be indented!

Most of the following configuration options are self-explaining. The default protocol is FTP; if you want to use WebDAV, please specify protocol webdav. The local directive contains the local path of the web site copy, remote contains the path of the web site on the remote server - it can be absolute or relative. If your user is chrooted (as is normally the case with FTP users), you should use a relative path (such as ~/ or ~/web). Otherwise use an absolute path.

The exclude lines are optional, they are here just to demonstrate how you can exclude files from being maintained by sitecopy.

You can find out more about sitecopy configuration on its man page:

man sitecopy

 

4 First Usage

Before you use sitecopy for the first time, you have to decide which of the following three scenarios matches your situation:

  1. Existing remote site and local copy, both in sync.
  2. Existing remote site, no local copy.
  3. New remote site, existing local copy.

 

4.1 Existing Remote Site and local copy, both in sync

If both the remote site and the local copy exist and are in sync, run

sitecopy --catchup example.com

to make sitecopy think the local site is exactly the same as the remote copy. Replace example.com with the name of the site you use in the .sitecopyrc file.

[email protected]:~$ sitecopy --catchup example.com
sitecopy: Catching up site `example.com' (on example.com in ~/web/)
sitecopy: All the files and and directories are marked as updated remotely.

 

4.2 Existing Remote Site, no local copy

If you have no local copy of the existing remote web site, run

sitecopy --fetch example.com

first so that sitecopy fetches the list of files from the remote server (replace example.com with the name of the site you use in the .sitecopyrc file):

[email protected]:~$ sitecopy --fetch example.com
sitecopy: Fetching site `example.com' (on example.com in ~/web/)
File: error/503.html - size 1906
File: error/502.html - size 1881
File: error/500.html - size 1851
File: error/405.html - size 1810
File: error/404.html - size 1806
File: error/403.html - size 1809
File: error/401.html - size 1806
File: error/400.html - size 1792
File: stats/.htaccess - size 128
File: robots.txt - size 14
File: index.html - size 1861
File: favicon.ico - size 7358
File: .htaccess - size 26
Directory: error/
Directory: stats/
sitecopy: Fetch completed successfully.

Then run

sitecopy --synch example.com

to update the local site from the remote copy.

sitecopy: Synchronizing site `example.com' (on example.com in ~/web/)
Creating error/: done.
Creating stats/: done.
Downloading error/503.html: [.] done.
Downloading error/502.html: [.] done.
Downloading error/500.html: [.] done.
Downloading error/405.html: [.] done.
Downloading error/404.html: [.] done.
Downloading error/403.html: [.] done.
Downloading error/401.html: [.] done.
Downloading error/400.html: [.] done.
Downloading stats/.htaccess: [.] done.
Downloading robots.txt: [.] done.
Downloading index.html: [.] done.
Downloading favicon.ico: [.] done.
Downloading .htaccess: [.] done.
sitecopy: Synchronize completed successfully.

 

4.3 New Remote Site, Existing Local Copy

If the local copy exists, but you have an empty remote site, run

sitecopy --init example.com

first to intialize the site. Replace example.com with the name of the site you use in the .sitecopyrc file.

[email protected]:~$ sitecopy --init example.com
sitecopy: Initializing site `example.com' (on example.com in ~/web/)
sitecopy: All the files and directories are marked as NOT updated remotely.

Then run:

sitecopy --update example.com

to upload the local copy to the remote site:

[email protected]:~$ sitecopy --update example.com
sitecopy: Updating site `example.com' (on example.com in ~/web/)
Creating stats/: done.
Creating data/: done.
Creating error/: done.
Uploading stats/.htaccess: [.] done.
Uploading data/index.html: [.] done.
Uploading error/403.html: [.] done.
Uploading error/401.html: [.] done.
Uploading error/404.html: [.] done.
Uploading error/503.html: [.] done.
Uploading error/400.html: [.] done.
Uploading error/502.html: [.] done.
Uploading error/405.html: [.] done.
Uploading error/500.html: [.] done.
Uploading index.html: [.] done.
Uploading robots.txt: [.] done.
Uploading .htaccess: [.] done.
Uploading favicon.ico: [.] done.
sitecopy: Update completed successfully.

 

5 Using sitecopy

Afterwards, sitecopy usage is really easy. You can work with your local copy and update, create, and delete files. A first, but optional step is to run

sitecopy example.com

to find out which files have changed locally (replace example.com with the name of the site you use in the .sitecopyrc file):

[email protected]:~$ sitecopy example.com
sitecopy: Showing changes to site `example.com' (on example.com in ~/web/)
* These items have been added since the last update:
info.php
sitecopy: The remote site needs updating (1 item to update).

To synchronize your remote web site with your local copy (i.e. upload new and changed files to the remote server and delete files on the remote server that have been deleted locally), you simply run

sitecopy --update example.com

[email protected]:~$ sitecopy --update example.com
sitecopy: Updating site `example.com' (on example.com in ~/web/)
Uploading info.php: [] done.
sitecopy: Update completed successfully.

That's it! Have fun with sitecopy!

 

Share this page:

5 Comment(s)

Add comment

Comments

From: SpartakusMd at: 2016-07-22 14:23:21

FYI : In case you have SSH access, the best option is to use GIT :)

From: dgrb at: 2016-07-23 17:05:58

Not sure what the advantage over wwwsync would be and in any case I'd be reluctant to use a package that relies on the unencrypted FTP for its transferring.

To keep my website up to date I use rsync over ssh. Easy to to, has the ability to exclude things and is secure.

From: till at: 2016-07-24 07:48:06

Not everyone runs its own server. You forget the millions of small websites hosted at ISP's that just offer FTP access and not offer SSH / rsync.

From: wirak() at: 2016-07-29 10:42:40

 Don't you think that sftp instead of ftp is a must?

From: till at: 2016-07-29 11:04:13

It depends on how and where you use sitecopy. If you use it in your local network or over a securely tunnelled connection to a server, then you don't have to encrypt the file transfers. Btw, SFTP is SSH and not FTP. If you refer to secured (encrypted FTP), then the protocol is named FTPS and not SFTP. Sitecopy supports WebDAV over HTTPS if your server has webdav support. I'm not the author of sitecopy, I just show you how to use it.