Build cache server exits with "permission denied" on its data

You start the container and it stops straight away. The log ends with permission denied. The cause I reproduced is who owns the data directory. This page shows the failure and the fix, reproduced for real with the 0.2.1 image.

What you see

The container exits with status 1 within a few seconds, and its log says:

{"level":"ERROR","msg":"fscache: fatal","error":"write shutdown marker: open data/.unclean-shutdown: permission denied"}

Why it happens

The server does not run as root. It runs as user 65532, and it keeps its files in a data folder inside /home/nonroot. If that folder already exists and belongs to someone else, the server cannot write to it and stops.

I reproduced it with a volume that already held data owned by root, which is what you get after restoring a backup as root:

/d       0:0   drwxr-xr-x      (owner:group, mode)
/d/data  0:0   drwxr-xr-x

The fix

Give the whole tree to user 65532. The image has no shell, so use a throwaway container to do it:

docker run --rm -v fscache-data:/d ubuntu:24.04 chown -R 65532:65532 /d

After that the same image started normally: /healthz answered ok and an upload returned 201.

What does not cause it

• A new, empty named volume. This is what our first-run steps create, and the volume then belongs to 65532. I also tried a new empty volume whose top folder I had set to root. The server still started, and afterwards the folder belonged to 65532, because Docker copies the image's ownership into an empty volume the first time it is used.

What I did not test

• A bind mount of a host folder on Linux. The same rule should apply (the folder has to be writable by 65532), but I ran on a Mac, where bind mounts behave differently.
• Kubernetes without the fsGroup setting. The manifests in our Kubernetes page set fsGroup: 65532, and I ran them with it; I did not run them without it.
• Running the container as root to avoid the problem. It is not what we document.

Related: back up and restore a build cache, where the restore step has to keep the ownership, and Gradle build cache with Docker Compose.