Skip tests that already passed with the Maven build cache
When nothing in a module has changed, the Maven build cache extension restores its result and skips the tests that already passed. At FosterStack we prove everything, so here is how we proved it to ourselves: real builds, the exact log lines, and the cases where the tests rightly ran again.
The short answer
- A fresh copy with the same code restored the whole build. No compile, no tests.
- Change the main code or the tests and Maven built the module and ran the tests again.
- In our runs, a failing test ran again each time, and a failing build was not restored.
What we ran
A one-module Maven project with one class and two JUnit tests, run with the Surefire plugin (3.5.2). Every build used a fresh copy of the project on the same Mac and an empty local build cache. The builds shared the cache server, and one Maven repository folder so the test libraries were downloaded once. All runs used mvn verify, with extra options for the cache location and the shared repository. The cache setup is the one on Maven remote build cache setup. We counted the Tests run: line that Maven prints.
| Run | Result | Tests run |
|---|---|---|
| 1. First build, as CI would do it | Built, BUILD SUCCESS | 2 |
| 2. Fresh copy, same code | Restored from the cache, BUILD SUCCESS | none |
| 3. Change the main code | Built, BUILD SUCCESS | 2 |
| 4. Add a test | Built, BUILD SUCCESS | 3 |
| 5. A test that fails | Built, BUILD FAILURE | 2, with 1 failure |
| 6. The same failing test, fresh copy | Built again, BUILD FAILURE | 2, with 1 failure |
In run 2 Maven restored the module and then listed the steps it skipped, the tests included:
[INFO] Found cached build, restoring demo:mtest from cache by checksum ...
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO] Skipping plugin execution (cached): resources:testResources
[INFO] Skipping plugin execution (cached): compiler:testCompile
[INFO] Skipping plugin execution (cached): surefire:test
[INFO] Skipping plugin execution (cached): jar:jarWhat this means for your builds
- The tests are skipped as part of restoring the whole module. Maven restores the module's result and then skips each step,
surefire:testamong them. In Gradle the test task is taken from the cache as an entry of its own; see Gradle test task FROM-CACHE. - A passing build was reusable, a failing one was not. In runs 5 and 6 the tests ran and failed each time. A fresh copy of the same broken code did not get a restored result.
- Each change we made rebuilt the module and ran all its tests. Run 3 changed one line of main code and both tests ran again. Run 4 added a test and all three ran. We did not try a comment-only edit or a change to a non-source file.
We did not test a project with several modules here. For that, see Maven build cache extension in a multi-module project. We also did not test tests that read the clock, the network or files outside the project: if a test result depends on something Maven does not see, a restored pass could stand when it should not. That is a decision about your tests, not about the cache.
If you want the tests to run every time
Turn the cache off or skip the lookup for that build. The switches, and what each did in a real run, are on enable or disable the Maven build cache extension.
If nothing is restored
See Maven build cache not restoring. The setup is on Maven remote build cache setup.
Run on Maven 3.10.0 with maven-build-cache-extension 1.2.3, Surefire 3.5.2, JUnit 5.11.0 and FosterStack Cache 0.2.1, macOS (arm64), Java 27.