Faster Gradle builds: skip tests whose code didn't change

When nothing a test depends on has changed, Gradle takes its passing result from FosterStack Cache instead of running it again, and your build finishes sooner. At FosterStack we prove everything, so here is how we proved it to ourselves: real runs, real numbers, and the cases where the tests rightly ran again.

The short answer

A fresh copy of the project with the same code got :test FROM-CACHE. The tests were not run.
In our runs 3 and 4, changing the main code or the test code made the tests run again.
A comment added on the same line recompiled the main code, but the tests still came from the cache.
In our runs, a failing test ran each time. It was never taken from the cache.

What we ran

A small Java project with one class and two JUnit tests. Every run used a fresh copy of the project on the same Mac, with the same Java and Gradle, and the local build cache was switched off. The runs shared the cache server, and one Gradle home folder so the test libraries were downloaded once. A different machine or Java version was not tested. All the runs used gradle test --console=plain --no-daemon.

Run	compileJava	compileTestJava	test
1. First build, as CI would do it	ran	ran	ran
2. Fresh copy, same code	FROM-CACHE	FROM-CACHE	FROM-CACHE
3. Change the main code	ran	FROM-CACHE	ran
4. Add a test	FROM-CACHE	ran	ran
5. Add a comment on the same line in the main code	ran	FROM-CACHE	FROM-CACHE
6. A test that fails	FROM-CACHE	ran	FAILED
7. The same failing test, fresh copy	FROM-CACHE	FROM-CACHE	FAILED

Runs 1 to 5 ended with BUILD SUCCESSFUL. Runs 6 and 7 ended with BUILD FAILED, as intended. This is the line run 2 printed:

> Task :test FROM-CACHE

How much time does it save?

On the first project the tests are so quick that the totals were 3 seconds and 2 seconds, which tells you nothing. So we made a second project, set up the same way as above, with three more tests that each wait 10 seconds, made slow on purpose so the time is visible. It still has the two quick tests. Each build used its own copy of the project and the local build cache was off. The Mac was busy with other jobs the whole time, so all the totals are high; the gap is what matters.

Run	`:test` task	Gradle's total
1. First build	ran	1m 23s
2. Fresh copy, same code	`FROM-CACHE`	35s
3. Main code changed	ran	1m 12s
4. Fresh copy, same code again	`FROM-CACHE`	32s

The builds where the tests came from the cache finished about 40 to 50 seconds sooner: 48 seconds sooner for run 2 against run 1, and 40 seconds sooner for run 4 against run 3. (Run 3 also recompiled the main code, so it is not the same two builds as run 1.) The three sleeping tests account for at least 30 seconds of that; the rest is compiling and starting the tests, which the cached builds skipped too. Even the fully cached builds took 32 to 35 seconds on this busy Mac. Each total is a single run. Your tests will differ, so measure your own build.

What this means for your builds

The first run pays, the next skips. After the tests have run once for a change, a fresh copy with the same code did not run them again (run 2). With CI and laptops that is the idea, though we ran everything on one Mac.
In our runs, only the steps whose inputs changed ran again. Changing test code left the main code's compile step in the cache (run 4). Changing the main code left the test code's compile step in the cache (run 3).
An edit that does not change the compiled output can still skip the tests (run 5). The comment made Gradle compile the main code again, but the compiled output evidently did not change, because the test task found its stored result. We only tried a comment on the same line; a comment that adds lines can change the compiled output, and we did not test that.
A failing test was not skipped. In runs 6 and 7 the test task ran and failed each time. In run 7, a fresh copy of the same broken code ran the tests again and reported 2 tests completed, 1 failed.

We did not test tests that read the clock, the network or files outside the project. If a test result depends on something Gradle does not see, a stored pass could be reused when it should not be. That is a decision about your tests, not about the cache.

The files we used

build.gradle.kts:

plugins { java }

repositories { mavenCentral() }

dependencies {
    testImplementation(platform("org.junit:junit-bom:5.11.0"))
    testImplementation("org.junit.jupiter:junit-jupiter")
    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
}

tasks.test { useJUnitPlatform() }

gradle.properties has one line, org.gradle.caching=true. settings.gradle.kts is the same cache setup as on enable or disable the Gradle build cache, with a different port. We used 127.0.0.1 over plain HTTP because the test ran on one machine; use HTTPS across a network.

If the test task does not come from the cache

Work through why the hit rate is low. For a multi-module build, see Gradle build cache for a monorepo.

Run on Gradle 9.8.0 with FosterStack Cache 0.2.1, macOS (arm64), Java 27, JUnit 5.11.0.