[HUDI-2983] Remove all Log4j2 transitive dependencies

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.1
• Labels: pull-request-available sev:critical
• Assignee: Unassigned
• Reporter: Udit Mehrotra
• Created: 11/Dec/21 01:35
• Updated: 26/Jan/22 13:34
• Resolved: 26/Jan/22 13:34

Description

Log4j2 is currently being bought in transitively by various dependencies like hive-exec, hive-service, flink test utils etc. This tracks removal of all of these.

 

[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-hadoop-mr ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-hadoop-mr:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:core:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] -----------------< org.apache.hudi:hudi-spark-client >------------------
[INFO] Building hudi-spark-client 0.11.0-SNAPSHOT                        [8/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-spark-client ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-spark-client:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:core:2.3.1:test
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO]
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-hive-sync ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-hive-sync:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-common:jar:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-spark_2.11 ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-spark_2.11:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:core:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] ----------------< org.apache.hudi:hudi-utilities_2.11 >-----------------
[INFO] Building hudi-utilities_2.11 0.11.0-SNAPSHOT                     [15/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-utilities_2.11 ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-utilities_2.11:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:core:2.3.1:test
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] -------------< org.apache.hudi:hudi-utilities-bundle_2.11 >-------------
[INFO] Building hudi-utilities-bundle_2.11 0.11.0-SNAPSHOT              [16/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-utilities-bundle_2.11 ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-utilities-bundle_2.11:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-common:jar:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO]
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-cli ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-cli:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-jdbc:jar:2.3.1:provided
[INFO]    \- org.apache.hive:hive-common:jar:2.3.1:provided
[INFO]       \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]          \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO]
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-java-client ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-java-client:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:core:2.3.1:test
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO]
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-flink-client ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-flink-client:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:core:2.3.1:test
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-dla-sync ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-dla-sync:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:2.3.1:compile
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:compile
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:compile
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-spark-bundle_2.11 ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-spark-bundle_2.11:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-common:jar:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] ------------------< org.apache.hudi:hudi-integ-test >-------------------
[INFO] Building hudi-integ-test 0.11.0-SNAPSHOT                         [39/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-integ-test ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-integ-test:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:core:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] ---------------< org.apache.hudi:hudi-integ-test-bundle >---------------
[INFO] Building hudi-integ-test-bundle 0.11.0-SNAPSHOT                  [40/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-integ-test-bundle ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-integ-test-bundle:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:2.3.1:compile
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:compile
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:compile
[INFO] 
[INFO] -------------------< org.apache.hudi:hudi-examples >--------------------
[INFO] Building hudi-examples 0.11.0-SNAPSHOT                           [41/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-examples ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-examples:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-common:jar:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] ------------------< org.apache.hudi:hudi-flink_2.11 >-------------------
[INFO] Building hudi-flink_2.11 0.11.0-SNAPSHOT                         [42/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-flink_2.11 ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-flink_2.11:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:core:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] -----------------< org.apache.hudi:hudi-kafka-connect >-----------------
[INFO] Building hudi-kafka-connect 0.11.0-SNAPSHOT                      [43/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-kafka-connect ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-kafka-connect:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-common:jar:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] 
[INFO] ---------------< org.apache.hudi:hudi-flink-bundle_2.11 >---------------
[INFO] Building hudi-flink-bundle_2.11 0.11.0-SNAPSHOT                  [44/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-flink-bundle_2.11 ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-flink-bundle_2.11:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-exec:jar:2.3.1:compile
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:compile
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:compile
[INFO] 
[INFO] -------------< org.apache.hudi:hudi-kafka-connect-bundle >--------------
[INFO] Building hudi-kafka-connect-bundle 0.11.0-SNAPSHOT               [45/45]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ hudi-kafka-connect-bundle ---
[INFO] Verbose not supported since maven-dependency-plugin 3.0
[INFO] org.apache.hudi:hudi-kafka-connect-bundle:jar:0.11.0-SNAPSHOT
[INFO] \- org.apache.hive:hive-common:jar:2.3.1:provided
[INFO]    \- org.apache.logging.log4j:log4j-1.2-api:jar:2.6.2:provided
[INFO]       \- org.apache.logging.log4j:log4j-core:jar:2.6.2:provided
[INFO] ------------------------------------------------------------------------

Issue Links

• links to: GitHub Pull Request #4281