Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.3.2
-
None
-
None
Description
We recently moved to new Macbooks with the M3 Max and have started seeing random failures where builds will fail trying to access files in target. Our project is a multi-module project with multiple sub-modules, with 50+ modules in total. For example...
pom.xml (parent)
module1 (parent is root)
pom.xml
sub-module1 (parent is module1
pom.xml
sub-sub-module (parent is sub-module1)
sub-module2 (parent is module1)
pom.xml
module2 (parent is root)
pom.xml
sub-module1 (parent is module1
pom.xml
sub-sub-module (parent is sub-module2)
sub-module2 (parent is module2)
pom.xml
module3, 4, 5, etc...
We build using -TC1, so on the M3 Max there are 16 threads used during the build.
We didn't see this problem often running on our older laptops; we rarely saw it on our Windows laptops because they only had 8 cores, but we would occasionally see it on our Macbook Pro 2019 models with i9 8 core (16 with hyperthreading). Now that we are using the new M3 Max we can't get a build to finish without failures.
We see that things fail randomly when trying to access files in the target. For example...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.6.1:analyze-only (analyze) on project authservice-webapp: Cannot analyze dependencies: /Users/username/workspace/trunk/common-resources/target/common-resources-28.0.0-SNAPSHOT.jar -> [Help 1]
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.6.1:copy-dependencies (copy-dependencies) on project spatialdata-loader: Error copying artifact from /Users/username/workspace/trunk/spatialdata/ant-spring-runner/target/ant-spring-runner-28.0.0-SNAPSHOT.jar to /Users/username/workspace//trunk/spatialdata/spatialdata-loader/target/lib/ant-spring-runner-28.0.0-SNAPSHOT.jar: File /Users/username/workspace/trunk/spatialdata/ant-spring-runner/target/ant-spring-runner-28.0.0-SNAPSHOT.jar does not exist ->
[Help 1]
To debug it down to possibly a threading issue related to using fast with maven-clean-plugin a tested the following...
If I build using "mvn clean install" the build always fails and in random places.
If I build with -T1 then the build never fails. It also doesn't fail with up to -T7, but failures start at -T8 or -TC1.
If I run mvn clean first, then mvn install, then the build never fails.
If I build using "mvn clean install", but I disable fast, then the build never fails.
This is our plugin configuration. All I have to do is set fast to false, and we won't see any of the random failures as I described above. So it seems like there is some threading issue with fast where files are being deleted.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-clean-plugin</artifactId>
<version>${maven-clean-plugin.version}</version>
<configuration>
<filesets>
<fileset>
<directory>${project.basedir}</directory>
<followSymlinks>true</followSymlinks>
<useDefaultExcludes>true</useDefaultExcludes>
<includes>
<include>*.tgz</include>
<include>src/**/generated-sources/**</include>
<include>dist/**</include>
<include>app/scripts/beans/**</include>
<include>src/scripts/beans/**</include>
<!-- <include>.tmp/</include>-->
</includes>
</fileset>
</filesets>
<verbose>false</verbose>
<failOnError>true</failOnError>
<retryOnError>true</retryOnError>
<fast>true</fast>
<fastDir>${project.build.directory}/.clean</fastDir>
<fastMode>background</fastMode>
</configuration>
</plugin>
One place it will randomly fail is a sub-module like this...
root
pom.xml
module A
pom.xml
sub-module 1
pom.xml
sub-module 2
pom.xml (sub-module 2 has a runtime dependency on module A/sub-module 1)
If I build module A by itself, then it never fails; it will build module A and all of its sub-modules without failures. But if I build the project's root, sub-module2 will almost always fails (90% of the time) when it tries to copy its runtime dependency from sub-module1 because the jar is missing from the target in sub-module1.
This is the related failure...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.6.1:copy-dependencies (copy-dependencies) on project sub-module2: Error copying artifact from /Users/username/workspace/trunk/moduleA/sub-module1/target/sub-module1-28.0.0-SNAPSHOT.jar to /Users/username/workspace/trunk/moduleA/sub-module2/target/lib/sub-module1-28.0.0-SNAPSHOT.jar: File /Users/username/workspace/trunk/moduleA/sub-module1/target/sub-module1-28.0.0-SNAPSHOT.jar does not exist
It could very likely be a bug in the reactor, but given that simply changing fast to false makes the problem go away, it seems to point the finger at the maven-clean-plugin.