It’s quite easy to set up a debian mirror. But having a mirror on a local server is rather overkill in a scenario where you simply regularly have say 3 boxes running some Debian testing amd64, 1 box running the same on arch i686 and 2 other boxes on Ubuntu. Well, it’s more caching than mirroring that you’ll want, as transparently (with no client side setup) as possible.
And that’s overly easy to do with nginx, similarly to Steam depot caching. No, really, just do the same!
So, assuming nginx and dnsspoof are already up and running -if not, really follow the link about steam cache- you want to:
– create the apt folders…
mkdir -p /srv/www/apt/debian /srv/www/apt/debian-security /srv/www/apt/ubuntu
chown www-data:www-data -R /srv/www/apt
cd /srv/www
ln -s /srv/www/apt/debian .
ln -s /srv/www/apt/debian-security .
ln -s /srv/www/apt/ubuntu .
– update nginx by adding a /etc/nginx/sites-available/apt (and a symlink in /etc/nginx/sites-enabled/) with:
# apt spoof/proxy
server {
listen 80;
server_name ftp.fr.debian.org security.debian.org fr.archive.ubuntu.com security.ubuntu.com;
access_log /var/log/nginx/apt.access.log;
error_log /var/log/nginx/apt.error.log;
root /srv/www/;
resolver 127.0.0.1;
allow 10.0.0.0/24;
allow 127.0.0.1;
deny all;
location /debian/pool/ {
try_files $uri @mirror;
}
location /debian-security/pool/ {
try_files $uri @mirror;
}
location /ubuntu/pool/ {
try_files $uri @mirror;
}
location / {
proxy_next_upstream error timeout http_404;
proxy_pass http://$host$request_uri;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded_For $proxy_add_x_forwarded_for;
add_header X-Mirror-Upstream-Status $upstream_status;
add_header X-Mirror-Upstream-Response-Time $upstream_response_time;
add_header X-Mirror-Status $upstream_cache_status;
}
location @mirror {
access_log /var/log/nginx/apt.remote.log;
proxy_store on;
proxy_store_access user:rw group:rw all:r;
proxy_next_upstream error timeout http_404;
proxy_pass http://$host$request_uri;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded_For $proxy_add_x_forwarded_for;
add_header X-Mirror-Upstream-Status $upstream_status;
add_header X-Mirror-Upstream-Response-Time $upstream_response_time;
add_header X-Mirror-Status $upstream_cache_status;
}
}
– add the new domains to be spoofed in /etc/dnsspoof.conf:
10.0.0.1 ftp.fr.debian.org
10.0.0.1 security.debian.org
10.0.0.1 fr.archive.ubuntu.com
10.0.0.1 security.ubuntu.com
Then you have to restart both nginx and dnsspoof. Obviously, the domains have to match the sources you have configured in /etc/apt/sources.list[.d] – should be the nearest hosts to your location.
And since you do not want to keep a complete archive, you need to add a cronjob to remove outdated files, like this /etc/cron.weekly/apt-cache:
#!/bin/sh
# cleanup apt mirrors:
# remove any file that has not been accessed in the last 30 days
find /srv/www/apt -type f -atime +30 -print0 | xargs -0 --no-run-if-empty rm
# remove any empty dir (except the main ones)
find /srv/www/apt -mindepth 2 -type d -empty -print0 | xargs -0 --no-run-if-empty rm -r
That’s it. You may notice that it actually only caches the pools, not the packages lists. It could work with the whole repositories, not just the pools, but I noticed some failures with packages lists checks from time to time and it’s not worth the trouble.
# cd /scratch 