[HDDS-7102] Exclude unnecessary RangerClient dependencies

• Type: Task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.0
• Component/s: build
• Labels: pull-request-available
• Target Version/s: 1.3.0
• Assignee: Siyao Meng
• Reporter: Attila Doroszlai
• Created: 06/Aug/22 12:05
• Updated: 18/Aug/22 06:16
• Resolved: 15/Aug/22 19:57

Description

Ranger brings as transitive dependency aws-java-sdk-bundle, which is a 254MB fat jar. Is it needed for Ozone-Ranger integration? Ozone already depends on AWS Java SDK, different version, non-fat jars.

size (MB)

   254 aws-java-sdk-bundle-1.12.125.jar
     1 aws-java-sdk-core-1.12.261.jar
     1 aws-java-sdk-kms-1.12.261.jar
     2 aws-java-sdk-s3-1.12.261.jar

Also, are all of the other dependencies (Kafka, Hive, Solr, etc.) needed? Can they be excluded, letting users provide them on an as-needed basis?

dependencies via Ranger

+- org.apache.ranger:ranger-intg:jar:2.3.0:compile
|  \- org.apache.ranger:ranger-plugins-common:jar:2.3.0:compile
|     +- commons-lang:commons-lang:jar:2.6:compile
|     +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
|     |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
|     |  \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
|     +- org.apache.ranger:ranger-plugins-cred:jar:2.3.0:compile
|     +- org.apache.ranger:ranger-plugins-audit:jar:2.3.0:compile
|     |  +- org.apache.kafka:kafka-clients:jar:2.8.1:compile
|     |  |  \- com.github.luben:zstd-jni:jar:1.4.9-1:compile
|     |  +- org.apache.solr:solr-solrj:jar:8.6.3:compile
|     |  |  \- org.slf4j:jcl-over-slf4j:jar:1.7.24:compile
|     |  +- org.eclipse.jetty:jetty-client:jar:9.4.31.v20200723:compile
|     |  +- org.apache.httpcomponents:httpmime:jar:4.5.6:compile
|     |  +- org.apache.httpcomponents:httpcore-nio:jar:4.4.6:compile
|     |  +- org.apache.httpcomponents:httpasyncclient:jar:4.1.3:compile
|     |  +- joda-time:joda-time:jar:2.10.6:compile
|     |  +- com.carrotsearch:hppc:jar:0.8.0:compile
|     |  +- org.apache.hive:hive-storage-api:jar:2.7.2:compile
|     |  +- org.apache.orc:orc-core:jar:1.5.8:compile
|     |  \- com.amazonaws:aws-java-sdk-bundle:jar:1.12.125:compile
|     +- net.java.dev.jna:jna:jar:5.2.0:compile
|     +- net.java.dev.jna:jna-platform:jar:5.2.0:compile
|     +- com.kstruct:gethostname4j:jar:0.0.2:compile
|     \- org.apache.ranger:ranger-plugin-classloader:jar:2.3.0:compile

Also, HDDS-5836 updated jar-report.txt, but not made any changes to the license. The CI check for new dependencies serves as a reminder to update the actual license, jar-report.txt is just a helper for that purpose.

Please update the hadoop-ozone/dist/src/main/license/bin/LICENSE.txt file with the modification
    AND execute hadoop-ozone/dist/src/main/license/update-jar-report.sh when you are ready

smeng, erose

Issue Links

• blocks: HDDS-7084 Release Ozone 1.3.0
• fixes: HDDS-7124 Setting logLevel through om logLevel endpoint broken
• is caused by: HDDS-5836 [Multi-Tenant] Use Ranger Java client
• links to: GitHub Pull Request #3669