[FLINK-20092] [Java 11] Multi-thread Flink compilation not working

• Type: Technical Debt
• Status: Closed
• Priority: Not a Priority
• Resolution: Fixed
• Component/s: Build System
• Labels: auto-deprioritized-major auto-deprioritized-minor pull-request-available
• Assignee: Unassigned
• Reporter: Maciej Bryński
• Created: 11/Nov/20 15:20
• Updated: 08/Oct/24 14:48
• Resolved: 16/Jan/23 13:22

Description

I'd like to use maven -T option when compiling flink.

 mvn -T 2C clean install -D"scala-2.12" -DskipTests

Unfortunately my build is stuck on:

[INFO] --- maven-shade-plugin:3.2.1:shade (shade-flink) @ flink-fs-hadoop-shaded ---
[INFO] Including org.apache.hadoop:hadoop-common:jar:3.1.0 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-annotations:jar:3.1.0 in the shaded jar.
[INFO] Including com.google.guava:guava:jar:11.0.2 in the shaded jar.
[INFO] Including commons-io:commons-io:jar:2.7 in the shaded jar.
[INFO] Including commons-collections:commons-collections:jar:3.2.2 in the shaded jar.
[INFO] Including commons-logging:commons-logging:jar:1.1.3 in the shaded jar.
[INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar.
[INFO] Including commons-beanutils:commons-beanutils:jar:1.9.3 in the shaded jar.
[INFO] Including org.apache.commons:commons-configuration2:jar:2.1.1 in the shaded jar.
[INFO] Including org.apache.commons:commons-lang3:jar:3.3.2 in the shaded jar.
[INFO] Including com.google.re2j:re2j:jar:1.1 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-auth:jar:3.1.0 in the shaded jar.
[INFO] Including org.apache.htrace:htrace-core4:jar:4.1.0-incubating in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-databind:jar:2.10.1 in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-annotations:jar:2.10.1 in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-core:jar:2.10.1 in the shaded jar.
[INFO] Including org.codehaus.woodstox:stax2-api:jar:3.1.4 in the shaded jar.
[INFO] Including com.fasterxml.woodstox:woodstox-core:jar:5.0.3 in the shaded jar.
[INFO] Including org.apache.flink:force-shading:jar:1.12-SNAPSHOT in the shaded jar.
[INFO] No artifact matching filter io.netty:netty
[WARNING] Discovered module-info.class. Shading will break its strong encapsulation.
[WARNING] Discovered module-info.class. Shading will break its strong encapsulation.
[WARNING] Discovered module-info.class. Shading will break its strong encapsulation.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /home/maverick/flink/flink-filesystems/flink-fs-hadoop-shaded/target/flink-fs-hadoop-shaded-1.12-SNAPSHOT.jar with /home/maverick/flink/flink-filesystems/flink-fs-hadoop-shaded/target/flink-fs-hadoop-shaded-1.12-SNAPSHOT-shaded.jar
[INFO] Dependency-reduced POM written at: /home/maverick/flink/flink-filesystems/flink-fs-hadoop-shaded/target/dependency-reduced-pom.xml

Can we make flink compilation working with multiple maven threads ?

Issue Links

• is blocked by: MSHADE-148 Shade Plugin gets stuck in infinite loop building dependency reduced POM
• Is contained by: FLINK-30083 Bump maven-shade-plugin to 3.4.1
• is duplicated by: FLINK-21499 maven package hang when using multithread on jdk 11
• is related to: FLINK-10725 Support for Java 11 (LTS)
• relates to: FLINK-25246 Make benchmarks runnable on Java 11
• relates to: FLINK-30083 Bump maven-shade-plugin to 3.4.1
• links to: GitHub Pull Request #25460