How To Create A Local Debian/Ubuntu Mirror With apt-mirror

Version 1.0
Author: Falko Timme

This tutorial shows how to create a Debian/Ubuntu mirror for your local network with the tool apt-mirror. Having a local Debian/Ubuntu mirror is good if you have to install multiple systems in your local network because then all needed packages can be downloaded over the fast LAN connection, thus saving your internet bandwidth.

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

 

1 Preliminary Note

You can set up a Debian and Ubuntu mirror on both Debian and Ubuntu systems, i.e., a Debian system can be a Debian and a Ubuntu mirror, as well as a Ubuntu system can be a Ubuntu and a Debian mirror.

Please note that you need a fast internet connection to download all the packages to your mirror as well as a lot of hard disk space. In my tests I mirrored the main, contrib, and non-free sections of Debian Sarge and the main, restricted, and universe sections of Ubuntu Edgy Eft which took about 25GB of hard disk space and about 6 hours of download time on a 16MBit DSL line.

I will not mirror security updates in this tutorial - I think it's a good idea to always download security updates directly from the internet. That way you can be sure you get the latest ones.

I've tested this on a Debian Sarge and a Ubuntu Edgy Eft system. If you use different Debian/Ubuntu versions, the procedure might differ a little bit. You should have already set up a basic Debian Sarge system (e.g. as outlined on pages one and two of https://www.howtoforge.com/perfect_setup_debian_sarge) or a basic Ubuntu Edgy Eft system (e.g. as outlined on pages one to three of https://www.howtoforge.com/perfect_setup_ubuntu_6.10).

In this example my local mirror has the IP address 192.168.0.100.

 

2 Install apt-mirror

2.1 Modify /etc/apt/sources.list (Debian Sarge only)

If you are on Debian Sarge, you must modify /etc/apt/sources.list to install apt-mirror. I don't know if this is necessary on Debian Etch or Sid, please try it yourself. This step is not necessary for Ubuntu Edgy Eft.

vi /etc/apt/sources.list

Please add the following line to the file:

[...]
deb http://apt-mirror.sourceforge.net/ apt-mirror/
[...]

Then update the packages database:

apt-get update

 

2.2 Install apt-mirror

This step is for all systems (Debian and Ubuntu). To install apt-mirror, please run

apt-get install apt-mirror

 

3 Install Apache

Later on, I want to access my new local Debian/Ubuntu mirror over http, therefore I install the Apache web server now:

apt-get install apache2

The default document root is /var/www, I will create symlinks from that directory to the location of the Debian and Ubuntu repository later on.

 

4 Configure apt-mirror

apt-mirror's configuration file is /etc/apt/mirror.list. There should already be a default configuration in it; on Debian Sarge the file looks like this:

# apt-mirror configuration file

##
## The default configuration options (uncomment and change to override)
##
#
# set base_path           /var/spool/apt-mirror
# set mirror_path  $base_path/mirror
# set skel_path           $base_path/skel
# set var_path     $base_path/var
#
# set defaultarch  
# set nthreads     20
#


##
## Example sources
##

# sarge's section
deb http://ftp.fi.debian.org/debian sarge main contrib non-free
deb-src http://ftp.fi.debian.org/debian sarge main contrib non-free

deb http://security.debian.org/debian-security sarge/updates main contrib non-free
deb-src http://security.debian.org/debian-security sarge/updates main contrib non-free

deb http://ftp.fi.debian.org/debian sarge main/debian-installer


# sarge-proposed-updates's section
deb http://ftp.fi.debian.org/debian sarge-proposed-updates main contrib non-free
deb-src http://ftp.fi.debian.org/debian sarge-proposed-updates main contrib non-free


# sid's section
deb http://ftp.fi.debian.org/debian sid main contrib non-free
deb-src http://ftp.fi.debian.org/debian sid main contrib non-free

deb http://ftp.fi.debian.org/debian sid main/debian-installer


# experimental section
deb http://ftp.fi.debian.org/debian ../project/experimental main contrib non-free
deb-src http://ftp.fi.debian.org/debian ../project/experimental main contrib non-free


##
## Cleaner configuration example
##
#
# set cleanscript $var_path/clean.sh
#

# Cleaning section
clean http://security.debian.org/
clean http://ftp.fi.debian.org/


skip-clean http://ftp.fi.debian.org/doc/
skip-clean http://ftp.fi.debian.org/tools/
skip-clean http://ftp.fi.debian.org/debian-cd/
skip-clean http://ftp.fi.debian.org/debian-minicd/
skip-clean http://ftp.fi.debian.org/debian/dists/sarge/main/installer-i386/
skip-clean http://ftp.fi.debian.org/debian/dists/sid/main/installer-i386/
skip-clean http://ftp.fi.debian.org/debian/doc/
skip-clean http://ftp.fi.debian.org/debian/tools/
skip-clean http://ftp.fi.debian.org/debian/project/
skip-clean http://ftp.fi.debian.org/debian-non-US/project/

On Ubuntu Edgy Eft, the file looks like this:

############# config ##################
#
# set base_path    /var/spool/apt-mirror
#
# if you change the base path you must create the directories below with write privlages
#
# set mirror_path  $base_path/mirror
# set skel_path    $base_path/skel
# set var_path     $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch  
set nthreads     20
set tilde 0
#
############# end config ##############

deb http://archive.ubuntu.com/ubuntu edgy main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu edgy-updates main restricted universe multiverse
#deb http://archive.ubuntu.com/ubuntu edgy-backports main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu edgy-security main restricted universe multiverse
#deb http://archive.ubuntu.com/ubuntu edgy-proposed main restricted universe multiverse

deb-src http://archive.ubuntu.com/ubuntu edgy main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu edgy-updates main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu edgy-backports main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu edgy-security main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu edgy-proposed main restricted universe multiverse

clean http://archive.ubuntu.com/ubuntu

If you compare the file with /etc/apt/sources.list, you will see that it's very similar. /etc/apt/mirror.list mainly lists the repositories that should be mirrored, plus a few other options.

Now we modify /etc/apt/mirror.list. As I said before, I want to mirror the main, contrib, and non-free sections of Debian Sarge and the main, restricted, and universe sections of Ubuntu Edgy Eft, but I don't want to mirror the security updates as these should be downloaded directly from the internet. Also, I don't want to mirror the source packages because in 95% of all installations you don't need source packages, and they need a lot of space on the hard disk. So my /etc/apt/mirror.list looks like this:

vi /etc/apt/mirror.list
# apt-mirror configuration file

##
## The default configuration options (uncomment and change to override)
##
#
# set base_path    /var/spool/apt-mirror
# set mirror_path  $base_path/mirror
# set skel_path    $base_path/skel
# set var_path     $base_path/var
#
# set defaultarch  
# set nthreads     20
#


##
## Example sources
##

# sarge's section
deb http://ftp.de.debian.org/debian sarge main contrib non-free

# Ubuntu Edgy Eft
deb http://de.archive.ubuntu.com/ubuntu/ edgy main restricted universe

##
## Cleaner configuration example
##
#
# set cleanscript $var_path/clean.sh
#

# Cleaning section
clean http://ftp.de.debian.org/
clean http://de.archive.ubuntu.com/

Please note that I used ftp.de.debian.org instead of ftp.fi.debian.org and de.archive.ubuntu.com instead of archive.ubuntu.com. You should use mirrors that are close to you, so if you are in the U.S., for example, replace de with us, and so on.

The cleaning section at the end tells apt-mirror in which directories on the local hard disk it should check to see if hard disk space can be freed. I want to check the whole local repositories for both Debian and Ubuntu, therefore I specify

clean http://ftp.de.debian.org/
clean http://de.archive.ubuntu.com/

If you want to use other Debian/Ubuntu versions, simply replace sarge with etch or sid and edgy with breezy, dapper, etc.

Share this page:

10 Comment(s)

Add comment

Please register in our forum first to comment.

Comments

By:

If you have both 32bit (i386) and 64 bit (amd64) you can have apt-mirror get the packages for both architectures. Just add the architecture after deb on each line.

Example:

deb-amd64 http://ftp.osuosl.org/pub/ubuntu gutsy main restricted universe multiverse

deb-i386 http://ftp.osuosl.org/pub/ubuntu gutsy main restricted universe multiverse

 

By: Anonymous

Thanks for the info. We just create a repo.

 

Thanks,
Ivu Loto Team

By: TCPMeta

I followed your guide to the T for the mirror portion. I am also using the mirror as a PXE server to load the debian install image. My clients do not like the mirror. I select HTTP and point it to my server's IP 192.168.1.100, point it to /debian/ and it asks what version I want and after I select Wheezy it fails. Might give it a shot with FTP next.

By: steven schuit

 Steven faggot loves to be lazy

By: Tom

My apt-mirror download finishes, but it still says that it needs to get 26.4 GiB, even after the script completes. Is this normal? I can keep re-running the script, but it finishes without downloading anything. In total I've downloaded 95GiB so far, it says it still has 26.4 GiB to download, but completes at 26.4 GiB.

 

26.4 GiB will be downloaded into archive.Downloading 394 archive files using 20 threads...Begin time: Tue Aug 23 08:20:22 2016[20]... [19]... [18]... [17]... [16]... [15]... [14]... [13]... [12]... [11]... [10]... [9]... [8]... [7]... [6]... [5]... [4]... [3]... [2]... [1]... [0]... End time: Tue Aug 23 08:26:18 2016

By: i386 and AMD64 architectures

I found the apt-mirror command would not replicate the folders for other CPU architectures - ie - installed a 16.10 64 bit system, and it only replicated the binary-amd64 folders. Apt-get update would complain the i386 folders were missing. I had to add

[ arch=amd64 ]

to the subscribing clients sources.list files.

 

 

By: jhebert

Hi! I have followed this process all the way through and am working on keeping a couple virtual servers up to date via a virtual lan switch. When I pull an "apt update" from the target machine I get the following:

Ign http://192.168.0.10 jessie InRelease

Ign http://192.168.0.10 jessie Release.gpg

Ign http://192.168.0.10 jessie Release

Err http://192.168.0.10 jessie/main i386 Packages 404 Not Found

Err http://192.168.0.10 jessie/contrib i386 Packages 404 Not Found

Err http://192.168.0.10 jessie/non-free i386 Packages 404 Not Found

Ign http://192.168.0.10 jessie/contrib Translation-en_US

Ign http://192.168.0.10 jessie/contrib Translation-en

Ign http://192.168.0.10 jessie/main Translation-en_US

Ign http://192.168.0.10 jessie/main Translation-en

Ign http://192.168.0.10 jessie/non-free Translation-en_US

Ign http://192.168.0.10 jessie/non-free Translation-en

W: Failed to fetch http://192.168.0.10/debian/dists/jessie/main/binary-i36/Packages 404 Not Found

W: Failed to fetch http://192.168.0.10/debian/dists/jessie/contrib/binary-i36/Packages 404 Not Found

W: Failed to fetch http://192.168.0.10/debian/dists/jessie/non-free/binary-i36/Packages 404 Not Found

E: Some index files failed to download they have been ignored, or old ones used instead.

 

Any suggestions would be very helpful!

By: sneha

Hey, how can I use https. How to use private IP for that purpose.

By: brun0

dpkg --remove-architecture i386

 

By: Adam Felson

I run ubuntu.  In order to mirror multiple repositories, I have one link, in /var/www/html:

lrwxrwxrwx 1 root root  28 Dec 29 11:52 mirror -> /var/spool/apt-mirror/mirror

I then updated the sources.list on my mirror user to be of the form: deb http://package-cache/mirror/archive.ubuntu.com/ubuntu/ disco restricted mainwhere package-cache is the hostname of my mirror.

I did a search and replace in sources.list to change http:// to http://package-cache/mirror/

An example of a non-ubuntu repo after the substitution is:

deb http://package-cache/mirror/apt.mopidy.com/ buster main contrib non-free