Some Tips On OpenVZ Deployment

I rely heavily on OpenVZ. In this article I would like to share some of my personal experiences in OpenVZ deployment. I assume that the readers already know how to install OpenVZ and the basics of OpenVZ. This article describes some tips on OpenVZ usage via the command line. If you prefer GUI to command line, please turn to how to install WebVZ.

The setup described here follows these guidelines:

  • the real server has minimum software installed (I use debian Etch with minimal installation) as the starting point. Additional applications are installed as needed along the deployment.
  • the real server should be as secure as possible. On the other hand, I want to keep it simple and easy to setup/maintain. So I chose a compromise: I rely only on what can be easily deployed with debian and don't go for extra security stuff like openwall, selinux, grsecurity, etc.
  • each needed service is deployed in a separate container, so that they interfere each other as little as possible
  • Intrusion Detection for the real server as well as the containers is deyployed on the real server using OSSEC
  • firewalling (iptables) is done on the real server; the containers run only the services
  • I rely on ssh as the only mean to access and maintain the real server and the containers.


Basic Security

Before deploying any ovz containers, I make some changes to the configuration of the real server to make it more secure:

  • disable root password
  • add a user admin-user who can sudo everything; this user has a simple password
  • add a user ssh-user who can ssh to the real server; after uploading ssh key for this user, I change the line in .ssh/authorized_keys to read:
    command="/bin/su - admin-user" ssh-rsa AAAA...
    and change /etc/pam.d/su to allow only this user to su:
    auth       required group=ssh-user
  • to copy files from/to the real server, I create another user sftp-user and install MySecureShell
  • change sshd_config so that:
    • only ssh-user and sftp-user are allowed to connect
    • clear password authentication is disable (after uploading ssh keys for ssh-user and sftp-user)
    • sshd runs on a non-standard port

The above scheme works as follows: to connect to the real server, we connect as ssh-user. Then we must type in password for admin-user. If someone gains ssh key for ssh-user, he still must know the password for admin-user in order to gain access to the server (failure of /bin/su - admin will immediately generate an email alert by OSSEC).

To copy files from/to the server, we use sftp with the account sftp-user. If someone gains ssh key for this user, this is not such a big problem, since he can access only files under his $HOME.


Creating OpenVZ Containers

I find it more comfortable to create a template for all containers, so that when I need a new container, I simply make make a clone from the template. I use only debian stable for the real server as well as for the containers. So, the first step is to create a template and tune it to my taste:

  • create a new container:
    vzctl create 2002 --ostemplate debian-4.0-amd64-minimal
  • set some basic parameters:
    vzctl set 2002 --ipadd --nameserver --hostname host2 --save
  • start the container:
    vzctl start 2002
  • enter the container:
    vzctl enter 2002
  • I prefer to keep everything at the bare minimum, and add stuff as needed. Since installing a package with debian is so easy, it takes very little effort to install any package we need. So I make the following changes for the template:
    • run aptitude and uncheck Options/Dependency handling/Install Recommended packages automatically
    • remove some packages that I don't want in the template:
    • edit /etc/apt/sources.list to tune it to my preferences
  • then I stop the template:
    vzctl stop 2002

Then anytime I need a new container, I use a script vz-clone as follows:


# script to clone an openvz VE

set -e

if [ -z "$2" ]; then
    echo "Usage: $0 <veid> <new-id>"
    exit 1


if [ ! -e $cfg ]; then 
    echo $cfg not found!
	exit 1

. $cfg

. $cfg

if [ -e $newcfg ]; then 
    echo $newcfg already exists!
	exit 1

if [ -e $newveprivate ]; then 
    echo $newveprivate already exists!
	exit 1

if vzlist | fgrep -w -q $1
    vzctl stop $1

echo "Cloning $cfg to $newcfg"
cp -a $cfg $newcfg

echo "Cloning $veprivate to $newveprivate"
mkdir -p $newveprivate
cd $veprivate
tar cf - . | (cd $newveprivate && tar xf -)

echo "Do not forget to edit $newcfg (you need to edit at least HOSTNAME and IP_ADDRESS)"
echo "Also do not forget to make an alias"


sudo sh vz-clone 2002 2010

Cloning /etc/vz/conf/2002.conf to /etc/vz/conf/2010.conf
Cloning /vz/private/2002 to /vz/private/2010
Do not forget to edit /etc/vz/conf/2010.conf (you need to edit at least HOSTNAME and IP_ADDRESS)
Also do not forget to make an alias

Depending on your /etc/vz/vz.conf, the paths in the above might be different. I use the below settings:


Then we need to edit /etc/vz/conf/2010.conf, change e.g. HOSTNAME to host10, IP_ADDRESS to and we are ready to go with the new container. We will also make an alias for the new container, which will be described in the next section.


Working With OpenVZ Containers

The ovz containers are identified by number. I find it easier to refer to them by name/alias, so that I don't have to remember for example 2010 is the id of the container running dns service. Apart from that, I also want to free myself from remembering the different commands vzctl, vzlist, vzquota, etc. and their parameters. So I create some simple scripts to help myself.

  • First I create a list of aliases /etc/vz-aliases:
    # aliases for openvz VE's
    2001  test
    2002  template
    2010  dns
    2020  ldap
    2030  mail
    2040  web
  • To translate between ID's and aliases, I create a script /usr/local/bin/vz-get-alias as below and make vz-get-veid as symlink to vz-get-alias:
    case $0 in
        cat $vz_alias_file | egrep "^[[:space:]]*$1[[:space:]]" | awk '{print $2}'
        cat $vz_alias_file | egrep "[[:space:]]$1[[:space:]]*$" | awk '{print $1}'
  • Then I put frequent commands to manipulate ovz containers to a script called /usr/local/bin/vz-cmd-generic:
    set -e
    ## handle vz-list first, since it requires no ID/alias
    case $0 in
    	cat /etc/vz-aliases | egrep '^[0-9]' | \
    	sed 's/\([0-9]*\) *\([a-zA-Z0-9-]*\)/s,\1 .*,\&\2,/' > $sedfile
    	sudo vzlist "[email protected]" | sed 's/                    $//' | \
    	sed -f $sedfile | \
    	sed '1s/$/ALIAS/'
    ## the other commands require an ID or alias 
    if [ -z "$1" ]; then
        echo "Usage: $0 <veid>|<alias> [<args>]"
    	exit 1
    veid=`/root/bin/vz-get-veid $1`
    if [ -z "$veid" ]; then
    case $0 in
        sudo vzctl start $veid
        sudo vzctl restart $veid
        sudo vzctl stop $veid
        sudo vzctl enter $veid
        sudo vzctl exec $veid "[email protected]"
        sudo vi /etc/vz/conf/$veid.conf
        sudo vzquota stat $veid
        sudo head -2 /proc/user_beancounters
    	sudo cat /proc/user_beancounters | egrep -A23 "^[[:space:]]+${veid}:"
    And make all the command vz-start, vz-stop, vz-exec, etc. as symlinks to this script vz-cmd-generic.

Usage is then simple:

  • to list all running containers:
       2010         15 running  host10      dns
    	  2020          8 running  host20      ldap
    	  2030         23 running  host30      mail
    	  2040         11 running  host40      web
  • to list all containers (including those which are not running):
    vz-list -a
       2002          - stopped   host2       template
       2010         15 running  host10      dns
    	  2020          8 running  host20      ldap
    	  2030         23 running  host30      mail
    	  2040         11 running  host40      web
  • to start/stop/restart a container:
    • vz-start dns
    • vz-stop dns
    • vz-restart dns
  • to exec a command inside a container:
    vz-exec dns aptitude update
  • to check UBC of a container:
    vz-ubc dns
  • to check quota of a container:
    vz-quota-ls dns
  • it is also possible to use an ID instead of an alias:
    vz-ubc 2010

It is also a good thing to keep the alias unique across different real servers, so that we can share /etc/vz-aliases between them without conflicts.

This article is already quite long, so let's stop here. We will continue in the next part, where we discuss issues like how to deploy Instrusion Detection with OSSEC, how to monitor and set UBC parameters for containers, etc.

Share this page:

Suggested articles

5 Comment(s)

Add comment




Thanks for the interesting article!

Not sure which version of vzctl you use, but since vzctl-3.0.11, released some time in Aug 2006, it supports calling VE by names, e.g.:

vzctl create 202 --ostemplate debian-4.0-i386-minimal --name mydns

vzctl start mydns

vzctl enter mydns

A generic way to obtain a VEID using its name is something like

vzlist -H -oveid $NAME

UBC parameters for the given container can be obtained from /proc/bc/$VEID/resources.

Hope that helps.


Also, regarding VE cloning you use -- you can achieve the same by packing the VE 2002 private area (/vz/private/2002) into a tarball and then using that tarball as a template, something similar to this:

tar czf /vz/template/cache/debian-4.0-amd64-mine.tar.gz -C /vz/private/2002

Next thing is to change the value of DEF_TEMPLATE in /etc/vz/vz.conf so you don't have to specify --ostemplate debian-4.0-amd64-mine each time you want to create a new VE based on your template.

Finally, a default config for new VE can be set using CONFIGFILE parameter in the same /etc/vz/vz.conf file.

Hope that helps.


thank you Kir for the great comments, it's a honour for me to receive them from the openvz developer himself.

When I started with openvz, alias was not supported so I made those shortcuts, which are not needed with newer version.

By: Brian

Just a quick note that the -C needs to be removed from the tar command for it to work.

By: amsayk

Newbie question, Doesn't using su - admin-user causes that password to be passed in the clear?

 Or is the connection encrypted continuously already given the key in authorized_keys?