Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
1.0.1, 1.1.0, 1.2.0
-
None
-
Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Maven home: C:\Users\sdkman\candidates\maven\current
Java version: 21.0.1, vendor: Eclipse Adoptium, runtime: C:\Users\sdkman\candidates\java\current
Default locale: en_US, platform encoding: UTF8
OS name: "windows 11", version: "10.0", arch: "amd64", family: "windows"Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae) Maven home: C:\Users\sdkman\candidates\maven\current Java version: 21.0.1, vendor: Eclipse Adoptium, runtime: C:\Users\sdkman\candidates\java\current Default locale: en_US, platform encoding: UTF8 OS name: "windows 11", version: "10.0", arch: "amd64", family: "windows"
-
Important
Description
We are trying to use the Maven build cache extension on a large multi-module project with ~180 modules by caching builds in the CI workflows to be able to reuse artifacts in the build cache between main and feature branches. The final build artifacts (i.e. jar, sources, apidocs) are used to build application Docker images and publish Maven modules into Nexus repository during deploy phase for releases.
Expected Behavior
When executing a build for a higher goal then the currently highest cached goal, the build extension should skip cached mojo executions, restore the cached artifacts (i.e. jar, javadocs, sources) into the project build directory and run remaining mojo executions for the increment, i.e. javadocs, sources, Docker images between verify -> deploy incremental build. After successful completion, the build cache info should be updated to record the new highest cached goal with incremental mojo executions and artifacts.
Current Behavior
When executing a build for a higher goal (i.e. deploy) then the currently highest cached goal (i.e. verify), the extension skips cached executions and runs mojos between cached and current goals while missing to restore cached final artifacts into the project build directory. After that, it runs the full build again from the begining to rebuild the artifacts and save build cache. Instead of reducing the build time by reusing already packaged artifacts and executions, it almost doubles the time to re-run the deploy for release from scratch. It also causes the Maven source plugin (3.3.0) to fail due to a duplicate sources artifact error, causing the deploy build to fail.
Possible Solution
It should be possible to restore cached artifacts into project build directory while avoiding to re-run full build again after restoring already cached mojo executions.
Steps to Reproduce
1. Run mvn clean package -DprojectVersion=1.1.0 from the tests/java/projects/build-extensions directory.
[INFO] Scanning for projects... [INFO] Loading cache configuration from C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\.mvn\maven-build-cache-config.xml [INFO] Using XX hash algorithm for cache [INFO] [INFO] -----------{-}< org.apache.maven.caching.test.simple:simple >{-}------------ [INFO] Building simple 0.0.1-SNAPSHOT [INFO] from pom.xml [INFO] -------------------------------{-}[ jar ]{-}-------------------------------- [INFO] [INFO] — clean:3.2.0:clean (default-clean) @ simple — [INFO] Going to calculate checksum for project [groupId=org.apache.maven.caching.test.simple, artifactId=simple] [INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system [INFO] Found 1 input files. Project dir processing: 12, plugins: 3 millis [INFO] Project inputs calculated in 30 ms. XX checksum [8e6f2406cb760579] calculated in 15 ms. [INFO] Attempting to restore project org.apache.maven.caching.test.simple:simple from build cache [INFO] Local build was not found by checksum 8e6f2406cb760579 for org.apache.maven.caching.test.simple:simple [INFO] [INFO] — resources:3.3.1:resources (default-resources) @ simple — [WARNING] Using platform encoding (UTF8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\src\main\resources [INFO] [INFO] — compiler:3.11.0:compile (default-compile) @ simple — [INFO] Changes detected - recompiling the module! :source [WARNING] File encoding has not been set, using platform encoding UTF8, i.e. build is platform dependent! [INFO] Compiling 1 source file with javac [debug target 1.8] to target\classes [WARNING] bootstrap class path not set in conjunction with -source 8 [WARNING] source value 8 is obsolete and will be removed in a future release [INFO] — compiler:3.11.0:testCompile (default-testCompile) @ simple — [INFO] No sources to compile [INFO] [INFO] — surefire:3.2.2:test (default-test) @ simple — [INFO] No tests to run. [INFO] [INFO] — jar:3.3.0:jar (default-jar) @ simple — [INFO] Building jar: C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target\simple-0.0.1-SNAPSHOT.jar [INFO] Saved Build to local file: C:\Users\.m2\build-cache\v1\org.apache.maven.caching.test.simple\simple\8e6f2406cb760579\local\buildinfo.xml [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4.679 s [INFO] Finished at: 2024-01-28T19:12:36-08:00 [INFO] ------------------------------------------------------------------------
2. Run mvn clean install -DprojectVersion=1.1.0
[INFO] Scanning for projects... [INFO] Loading cache configuration from C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\.mvn\maven-build-cache-config.xml [INFO] Using XX hash algorithm for cache [INFO] [INFO] -----------{-}< org.apache.maven.caching.test.simple:simple >{-}------------ [INFO] Building simple 0.0.1-SNAPSHOT [INFO] from pom.xml [INFO] -------------------------------{-}[ jar ]{-}-------------------------------- [INFO] [INFO] — clean:3.2.0:clean (default-clean) @ simple — [INFO] Deleting C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target [INFO] Going to calculate checksum for project [groupId=org.apache.maven.caching.test.simple, artifactId=simple] [INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system [INFO] Found 1 input files. Project dir processing: 6, plugins: 3 millis [INFO] Project inputs calculated in 24 ms. XX checksum [8e6f2406cb760579] calculated in 15 ms. [INFO] Attempting to restore project org.apache.maven.caching.test.simple:simple from build cache [INFO] Local build found by checksum 8e6f2406cb760579 [INFO] Found cached build, restoring org.apache.maven.caching.test.simple:simple from cache by checksum 8e6f2406cb760579 [INFO] Project org.apache.maven.caching.test.simple:simple restored partially. Highest cached goal: package, requested: install [INFO] Skipping plugin execution (cached): resources:resources [INFO] Skipping plugin execution (cached): compiler:compile [INFO] Skipping plugin execution (cached): resources:testResources [INFO] Skipping plugin execution (cached): compiler:testCompile [INFO] Skipping plugin execution (cached): surefire:test [INFO] Skipping plugin execution (cached): jar:jar [INFO] [INFO] — install:3.1.1:install (default-install) @ simple — [INFO] Installing C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\pom.xml to C:\Users\.m2\repository\org\apache\maven\caching\test\simple\simple\0.0.1-SNAPSHOT\simple-0.0.1-SNAPSHOT.pom [INFO] Installing C:\Users\.m2\build-cache\v1\org.apache.maven.caching.test.simple\simple\8e6f2406cb760579\local\simple.jar to C:\Users\.m2\repository\org\apache\maven\caching\test\simple\simple\0.0.1-SNAPSHOT\simple-0.0.1-SNAPSHOT.jar [INFO] [INFO] — resources:3.3.1:resources (default-resources) @ simple — [WARNING] Using platform encoding (UTF8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\src\main\resources [INFO] [INFO] — compiler:3.11.0:compile (default-compile) @ simple — [INFO] Changes detected - recompiling the module! :source [WARNING] File encoding has not been set, using platform encoding UTF8, i.e. build is platform dependent! [INFO] Compiling 1 source file with javac [debug target 1.8] to target\classes [WARNING] bootstrap class path not set in conjunction with -source 8 [WARNING] source value 8 is obsolete and will be removed in a future release [WARNING] target value 8 is obsolete and will be removed in a future release [WARNING] To suppress warnings about obsolete options, use -Xlint:-options. [INFO] [INFO] — resources:3.3.1:testResources (default-testResources) @ simple — [WARNING] Using platform encoding (UTF8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\src\test\resources [INFO] [INFO] — compiler:3.11.0:testCompile (default-testCompile) @ simple — [INFO] No sources to compile [INFO] [INFO] — surefire:3.2.2:test (default-test) @ simple — [INFO] No tests to run. [INFO] [INFO] — jar:3.3.0:jar (default-jar) @ simple — [INFO] Building jar: C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target\simple-0.0.1-SNAPSHOT.jar [INFO] [INFO] — install:3.1.1:install (default-install) @ simple — [INFO] Installing C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\pom.xml to C:\Users\.m2\repository\org\apache\maven\caching\test\simple\simple\0.0.1-SNAPSHOT\simple-0.0.1-SNAPSHOT.pom [INFO] Installing C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target\simple-0.0.1-SNAPSHOT.jar to C:\Users\.m2\repository\org\apache\maven\caching\test\simple\simple\0.0.1-SNAPSHOT\simple-0.0.1-SNAPSHOT.jar [INFO] Saved Build to local file: C:\Users\.m2\build-cache\v1\org.apache.maven.caching.test.simple\simple\8e6f2406cb760579\local\buildinfo.xml [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.956 s [INFO] Finished at: 2024-01-28T19:13:10-08:00 [INFO] ------------------------------------------------------------------------
3. Run mvn clean install -DprojectVersion=1.1.0
[INFO] Scanning for projects... [INFO] Loading cache configuration from C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\.mvn\maven-build-cache-config.xml [INFO] Using XX hash algorithm for cache [INFO] [INFO] -----------{-}< org.apache.maven.caching.test.simple:simple >{-}------------ [INFO] Building simple 0.0.1-SNAPSHOT [INFO] from pom.xml [INFO] -------------------------------{-}[ jar ]{-}-------------------------------- [INFO] [INFO] — clean:3.2.0:clean (default-clean) @ simple — [INFO] Deleting C:\Users\git\igdianov\maven-build-cache-extension\src\test\projects\build-extension\target [INFO] Going to calculate checksum for project [groupId=org.apache.maven.caching.test.simple, artifactId=simple] [INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system [INFO] Found 1 input files. Project dir processing: 6, plugins: 3 millis [INFO] Project inputs calculated in 24 ms. XX checksum [8e6f2406cb760579] calculated in 13 ms. [INFO] Attempting to restore project org.apache.maven.caching.test.simple:simple from build cache [INFO] Local build found by checksum 8e6f2406cb760579 [INFO] Found cached build, restoring org.apache.maven.caching.test.simple:simple from cache by checksum 8e6f2406cb760579 [INFO] Skipping plugin execution (cached): resources:resources [INFO] Skipping plugin execution (cached): compiler:compile [INFO] Skipping plugin execution (cached): resources:testResources [INFO] Skipping plugin execution (cached): compiler:testCompile [INFO] Skipping plugin execution (cached): surefire:test [INFO] Skipping plugin execution (cached): jar:jar [INFO] Skipping plugin execution (cached): install:install [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 0.667 s [INFO] Finished at: 2024-01-28T19:18:51-08:00 [INFO] ------------------------------------------------------------------------
4. Check target directory contexts: ls target/ -la
drwxr-xr-x 1 igdianov 197121 0 Jan 28 19:18 . drwxr-xr-x 1 igdianov 197121 0 Jan 28 19:18 .. drwxr-xr-x 1 igdianov 197121 0 Jan 28 19:18 maven-incremental
Context (Environment)
Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
Maven home: C:\Users\sdkman\candidates\maven\current
Java version: 21.0.1, vendor: Eclipse Adoptium, runtime: C:\Users\sdkman\candidates\java\current
Default locale: en_US, platform encoding: UTF8
OS name: "windows 11", version: "10.0", arch: "amd64", family: "windows"
Attachments
Issue Links
- links to