Apt-larder — an APT caching proxy written in Crystal

News
1

I’ve been running Debian/Ubuntu machines in a homelab and in CI for a while, and re-downloading the same .deb files over and over got old. So I wrote apt-larder — an HTTP caching proxy for APT repositories, in Crystal.

stock your packages, serve them fresh.

It sits between apt clients and upstream mirrors: .deb packages are cached indefinitely, index files (Release, Packages, InRelease, …) for a configurable TTL.

Usage

Two modes, which can coexist. Either as a plain HTTP proxy:

# /etc/apt/apt.conf.d/01proxy
Acquire::http::Proxy "http://apt-larder-host:3142";

…or by embedding the upstream host in the path, if you’d rather not touch the APT proxy config:

deb http://apt-larder-host:3142/deb.debian.org/debian trixie main

No configuration is required to get started — immutability is inferred from the URL structure (.deb/.udeb/.ddeb, /pool/, /by-hash/), so there’s no per-distribution setup.

What’s in it

  • Single-flight deduplication — concurrent requests for the same file trigger exactly one upstream fetch; the other fibers wait on a Channel.
  • Integrity — a .sha256 sidecar per cached file; immutable files are verified on first serve, corrupt ones are invalidated and re-fetched. Incomplete downloads (Content-Length mismatch) are never cached.
  • Range requests — 206 Partial Content, so interrupted downloads resume.
  • Per-host connection pool with stale-connection retry, conditional GET (If-Modified-Since) for index revalidation.
  • LRU eviction by age and/or total cache size.
  • Admin server (optional, separate port): a small web UI (dashboard + cache browser) and a JSON REST API, including /api/metrics in Prometheus format. A Grafana dashboard ships in extra/.
  • systemd Type=notify — READY=1 once bound, STOPPING=1 on SIGTERM before draining in-flight requests, watchdog ping, and hourly stats in STATUS= so systemctl status shows the hit rate. SIGUSR1 reopens the log file for logrotate.
  • Ships as a fully-static Linux binary (amd64 + arm64) and a distroless Docker image.

Crystal-specific notes

The whole thing is stdlib plus admiral for the CLI, tallboy for table output and spectator for the specs. Fibers made the single-flight and the graceful-drain logic pleasantly boring to write, and HTTP::Client handling the http→https redirect upgrade transparently saved a good chunk of work.

One thing I’m waiting on: Socket#sendfile (#16665, milestone 1.21.0). The hot HIT path is currently a 64 KB buffered copy loop; once 1.21 ships that becomes a zero-copy serve.

There’s a comparison with apt-cacher-ng in the README for anyone wondering why not just use that — short version: apt-cacher-ng is more mature and far more configurable, apt-larder trades that flexibility for a single binary with no config.

Source, releases and docs: GitHub - jbox-web/apt-larder: apt-larder - stock your packages, serve them fresh. · GitHub (MIT). Feedback and issues very welcome.