September 3rd, 2024

Setting up a cache server for apt packages

DJ Adams established an apt-cacher-ng cache server on a repurposed Asus Chromebox to minimize data usage for Debian packages, emphasizing security and monitoring for efficient package management.

Read original articleLink Icon
Setting up a cache server for apt packages

DJ Adams shares his experience setting up an apt-cacher-ng cache server for Debian packages in a home operations context, particularly while living on a narrowboat. The motivation for this setup stems from a desire to minimize data consumption over his internet connection, despite having an unlimited data plan. After transitioning from a Raspberry Pi to a repurposed Asus Chromebox running Proxmox, he created two containers: one for Docker and another for caching. The apt-cacher-ng service was installed on the caching container to serve as a central repository for Debian packages, which would reduce the need to repeatedly download the same packages. The setup process involved creating a Debian-based container, installing apt-cacher-ng, and configuring client containers to use the cache server. The author also highlights the importance of security settings during installation and provides insights into monitoring cache usage through log files. The successful installation and configuration allowed for efficient package management, demonstrating the benefits of using a caching proxy in a resource-conscious environment.

- DJ Adams set up an apt-cacher-ng cache server to reduce data usage for Debian packages.

- The server was created on a repurposed Asus Chromebox running Proxmox.

- Configuration involved creating client containers that point to the cache server for package management.

- The setup process emphasized security and monitoring through log files.

- The caching solution proved effective in managing package installations efficiently.

Link Icon 5 comments
By @simmons - 5 months
I'm a long-time user of apt-cacher-ng, but reading this reminds me of some of the pain points I regularly experience. Maybe others have some thoughts.

It would be nice if my Docker image builds, which may include apt-get steps, could benefit from the cache. I know Docker build will cache layers itself, but this doesn't check the upstream for fresher packages in the same way that could be done with HTTP caching. I know I could simply set the Acquire::http::Proxy in the Dockerfile, but then I've mixed local infrastructure concerns into a Dockerfile that should be generically usable by anyone, anywhere. It would be great if there were some way to inject these site-specific configurations into the image without tampering with the Dockerfile. This could be tricky, since the base image of any random Docker image isn't even guaranteed to be Debian. (Although I could imagine a very generic Bourne shell script that consumes /etc/os-release, if present, and performs any distro-specific customization.) This would also solve the similar problem of needing to inject site-specific trusted enterprise CA certificates into images.

Another pain point is the lack of HTTPS caching, which the author mentions. I'm not sure that dropping down to plain HTTP is the solution. I sometimes wonder if there could be a MitM proxy approach, where the cache presents a certificate for the remote hostname that is trusted by a CA certificate installed on the client. (In other words, something similar to what a Zscaler does to intercept HTTPS.)

By @rwky - 5 months
Personally I prefer to do `deb http://HTTPS///get.docker.com/ubuntu docker main` for HTTPS repos you get the benefit of caching that way [1]

Adding apt-cacher-ng is also a good thing for CI/CD if you add it to your build servers and point your docker builds to it you'll save bandwidth and build time.

1. https://www.unix-ag.uni-kl.de/~bloch/acng/html/howtos.html#s...

By @bravetraveler - 5 months
For those interested in running infrastructure like this but for more software, look at Pulp: https://github.com/pulp
By @mkesper - 5 months
About the needed apt https support: Finally, up to date builds of docker.io are coming back to Debian! https://tracker.debian.org/pkg/docker.io