Production build cache: Docker Compose, nginx and TLS

A Compose file for a shared build cache that only answers over HTTPS, with a password, behind nginx. Everything here was run for real on FosterStack Cache 0.2.1 and Gradle 9.8.0. The exact list of what was and was not tested is just below.

What was tested, and what was not

Tested: the two files below, with Docker Compose on a Mac, nginx 1.29 as the proxy, a self-signed certificate for localhost, and a real Gradle build over HTTPS.
Not tested: a certificate from a public authority or automatic renewal, a public server on port 443, other proxies or load balancers, Kubernetes ingress, and long-running use.

compose.yaml

services:
  fscache:
    image: ghcr.io/fosterstack/cache:0.2.1
    restart: unless-stopped
    # no "ports:" on purpose: only the proxy is reachable from outside
    volumes:
      - fscache-data:/home/nonroot
    environment:
      FSCACHE_MAX_BYTES: "53687091200"      # 50 GiB
      FSCACHE_USERNAME: gradle
      FSCACHE_PASSWORD: change-me           # generate one: openssl rand -base64 24

  proxy:
    image: nginx:1.29-alpine
    restart: unless-stopped
    depends_on:
      - fscache
    ports:
      - "127.0.0.1:8443:443"                # use 443:443 on a real server
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
      - ./certs:/etc/nginx/certs:ro

volumes:
  fscache-data:

The cache has no ports entry on purpose. Only the proxy is reachable from outside. Generate a real password, for example with openssl rand -base64 24, and never keep change-me.

nginx.conf

server {
    listen 443 ssl;
    server_name localhost;
    ssl_certificate     /etc/nginx/certs/cert.pem;
    ssl_certificate_key /etc/nginx/certs/key.pem;

    # the cache accepts entries up to 1 GiB by default; nginx's own default is 1 MB
    client_max_body_size 1g;

    location / {
        proxy_pass http://fscache:8080;
        proxy_set_header Host $host;
    }
}

The client_max_body_size line matters. See the tests below.

What I checked, and what I saw

Check	Result
`https://localhost:8443/healthz` with the certificate trusted	`ok`
The same without trusting the certificate	curl refuses (exit 60)
The cache's own port 8080 from the host	not reachable
Plain `http://` to the proxy's TLS port	400
Upload with no password, through the proxy	401
Upload with the password, through the proxy	201, and the value reads back
Uploads of about 1 MB and 2 MB with nginx's default settings	413 from nginx, not from the cache
The same uploads, and 20 MB, after `client_max_body_size 1g;`	201 each

The size limit is the one to watch. nginx rejects bodies over 1 MB by default, and Gradle's cache entries are often bigger, so uploads of larger entries would be refused. The cache itself accepts up to 1 GiB per entry by default, which is why the example uses 1g. See the status codes page.

Gradle over HTTPS

With a certificate Gradle does not trust, the build still succeeds, but without the cache, and prints:

Could not load entry d4f2de693b561fd2c5d60b17a9232d6f from remote build cache: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

For my self-signed test certificate I made Java trust it:

keytool -importcert -alias fsprod -file certs/cert.pem -keystore trust.jks -storepass changeit -noprompt

# gradle.properties
systemProp.javax.net.ssl.trustStore=/path/to/trust.jks
systemProp.javax.net.ssl.trustStorePassword=changeit

With that, the first build ran :compileJava and the second printed > Task :compileJava FROM-CACHE, through the proxy, over HTTPS, with the password. With a certificate from an authority Java already trusts you would not need the truststore lines. The Gradle side of the setup is in the tutorial.

Your first 15 minutes FosterStack home