[MSHADE-329] Concurrent writes of dependency-reduced-pom.xml adds same exclusion ad infinitum

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.2.1
• Assignee: Unassigned
• Reporter: Håkon Hallingstad
• Created: 13/Sep/19 15:52
• Updated: 15/Jan/21 05:00

Description

I have a multi-threaded mvn install that seems to halt but ends up using 200% CPU, in about 50% of the invocations. The mvn command used is:

mvn -T1C -nsu -Dmaven.source.skip -Dmaven.javadoc.skip -Dmaven.test.skip install -rf :MODULE

Using mvnDebug I have found out that there are 2 running Java threads, each writing dependency-reduced-pom.xml in two different modules A and B, respectively. These files seems to become several MB large, before they're deleted and then written again, and so forth.

I have looked into one of the threads, and there is a rewriteDependencyReducedPomIfWeHaveReduction in ShadeMojo with a loopCounter with value 2735, that just keeps increasing. Presumably there is something like one dependency-reduced-pom.xml written per iteration.

From the source code it seems this can only happen if https://github.com/apache/maven-shade-plugin/blob/master/src/main/java/org/apache/maven/plugins/shade/mojo/ShadeMojo.java#L1172 is hit at least that number of times, meaning the ShadeMojo adds that many exclusions, which seems to correspond to hamcrest-core:

 <dependencies>
  <dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
      <version>4.12</version>
     <scope>test</scope>
    <exclusions>
      <exclusion>
        <artifactId>hamcrest-core</artifactId>
        <groupId>org.hamcrest</groupId>
      </exclusion>
      <exclusion>
        <artifactId>hamcrest-core</artifactId>
        <groupId>org.hamcrest</groupId>
      </exclusion>
     ...

grep hamcrest-core dependency-reduced-pom.xml | wc -l
2735

 

The same exclusion is added loopCounter times in updateExcludesInDeps.

There is no problem running mvn in single-thread mode.

Issue Links

• relates to: MSHADE-124 Need better plan for getting dependency-reduced-pom.xml out of basedir
• relates to: FLINK-14318 JDK11 build stalls during shading