S3-backed CI cache helper. A single static Go binary.
cacher is the build that the typical "check S3, fall back to
upstream, upload for next time" shell loop should have been. It downloads,
uploads, lists, and invalidates cached artifacts in any S3-compatible
bucket — single files, docker images (via streamed
save | zstd | s3), and whole directory trees keyed by lockfile
hash. Built for builds.sr.ht; works
anywhere you can run a binary and reach an S3 endpoint.
Pre-built binaries for v0.1.2. The shipping URL is stable; the SHA-256s below are specific to this build — paste them into your pin if you care.
| Platform | Binary | SHA-256 |
|---|---|---|
| linux/amd64 | cacher-linux-amd64 | 95d02052b4125a4f4edaea4a0d7119aa589e01dbc4baa8778cfd2faf3ecec932 |
| linux/arm64 | cacher-linux-arm64 | 685df53170294c766cbcfbb2a5f0c1647c6cf8e6af4beac83c47007088dfaa51 |
| darwin/amd64 | cacher-darwin-amd64 | 6b6c3d97455b118eddcd7e41ad40e8b847af14448ee0d62229f2dd5f34f5d2d6 |
| darwin/arm64 | cacher-darwin-arm64 | 9ab5cb841bcb9de64445f3315b76c8d8f788fe26befe0d09e1e133cbf9717e26 |
All four hashes plus filenames are also available in a single
checksums.txt for piping into sha256sum -c:
wget https://bigbes.pages.srht.bigb.es/ci-cacher/cacher-linux-amd64 \
-O ~/.local/bin/cacher
chmod +x ~/.local/bin/cacher
wget -qO- https://bigbes.pages.srht.bigb.es/ci-cacher/checksums.txt \
| sha256sum -c --ignore-missing# before — install awscli, write ~/.aws/config, then in every task:
if aws s3api head-object --bucket "$B" --key "$K" >/dev/null 2>&1; then
aws s3 cp "s3://$B/$K" "$out"
else
curl -sSL "$url" -o "$out"
aws s3 cp "$out" "s3://$B/$K"
fi# after — one binary, one config, one command:
cacher download "$key" "$out" --url "$url"init / doctor — persist config + smoke-test credsdownload / upload — single file, with --url fallback and --sha256 verifyexists / list / delete / key — management + shell helpersdocker {exists,download,upload} — streamed save/load via zstddir {download,upload} — tar+zstd directory caching, keyed by content hash
Run cacher --help for the full surface. Read
the
README for usage patterns, Garage compatibility notes, and exit-code semantics.
The shell version this replaced repeated five things in every CI task:
install AWS CLI v2 (≈50 MB per build), write a Garage-tuned
~/.aws/config, compute cache keys from file content with
sha256sum | cut, branch HIT/MISS by hand, and for
docker images pipe docker save | zstd | aws s3 cp - (and the
inverse). cacher collapses all of that into one fetched binary
plus a config file. The directory caching is the genuinely new capability —
the shell version only ever handled single files.
All notable changes to this project will be documented in this file. The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
max-width from 44rem to 55rem.cacher docker download --pull — on cache miss, falls back to
docker pull <image:tag> and seeds the S3 cache. Mirrors the
--url fallback on file download; collapses the if/else
cache-or-pull bash dance in CI manifests to a single command.First public release. Replaces the s3_cache_or_curl /
s3_cache_docker_image shell helpers in tarantool-protobuf/.builds/lib/ci-lib.sh
with a single static Go binary.
cacher init / cacher doctor — persist config to
~/.config/cacher/config.toml and smoke-test S3 credentials (HEAD bucket
cacher download / upload / exists / list / delete for single
files. Download falls back to --url on cache miss and back-fills the
cache. Optional --sha256 verifies the fetched content.cacher docker {exists,download,upload} — streamed docker save | zstd
→ S3 multipart upload (and inverse). Pure-Go zstd via
klauspost/compress, no external
zstd binary on the host.cacher dir {download,upload} — tar+zstd of a directory tree keyed by
content hash. Closes the gap left by the shell version, which only
cached single files.cacher key — resolve a key template (substituting {hash}) for shell
scripting.--hash-from <path> (repeatable; files or directories) on every command.
For a single file path the digest exactly matches sha256sum file | cut -c1-N,
so existing keys migrate without recomputation.--arch-suffix opt-in to suffix every key with -<goos>-<goarch>.list --recursive for flat listing; list --root to ignore the
configured prefix and list at bucket root. Default output style mirrors
aws s3 ls (delimited by /).when_required (Garage
doesn't implement boto3 1.36+ trailing CRC32 checksums).go test -tags=e2e) against a real Garage
container via testcontainers-go using dxflrs/garage:v2.3.0's
--single-node --default-bucket mode.unit.yml (every push), e2e.yml (every push, with
Docker), publish.yml (tags only, ships cross-platform binaries via
goreleaser to pages.sr.ht and as build artifacts).doctor uses ListObjectsV2 (1-key) instead of HeadBucket — Garage
rejects HeadBucket with 403 even for valid credentials.~/.config/cacher/config.toml on every platform
(previously fell into ~/Library/Application Support/cacher on macOS
via os.UserConfigDir). cacher is a CI tool; dev-macs and Linux runners
must look in the same place.