[HADOOP-15265] Exclude json-smart explicitly in hadoop-auth avoid being pulled in transitively

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.1.0
• Hadoop Flags: Reviewed
• Assignee: Nishant Bangarwa
• Reporter: Nishant Bangarwa
• Created: 26/Feb/18 19:15
• Updated: 26/Feb/18 22:21
• Resolved: 26/Feb/18 21:58

Description

this is an extension of - https://issues.apache.org/jira/browse/HADOOP-14903

We need to exclude the dependency explicitly in hadoop-auth pom.xml and add the correct version so that it is not being pulled transitively. 

In Druid we use, https://github.com/tesla/tesla-aether/blob/master/src/main/java/io/tesla/aether/TeslaAether.java to fetch dependencies transitively, which is still pulling in wrong version of json-smart jar.

org.apache.hadoop:hadoop-auth:jar:2.7.3.2.6.5.0-129 -> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 -> net.minidev:json-smart:jar:2.3-SNAPSHOT

 

Full Stack trace 

 2018/02/26 03:47:22 INFO    : 2018-02-26T03:47:22,878 ERROR [main] io.druid.cli.PullDependencies - Unable to resolve artifacts for [io.druid.extensions:druid-hdfs-storage:jar:0.10.1.2.6.5.0-129 (runtime) -> [] < [ (https://repo1.maven.org/maven2/, releases+snapshots),  (http://nexus-private.hortonworks.com/nexus/content/groups/public, releases+snapshots),  (http://nexus-private.hortonworks.com/nexus/content/groups/public, releases+snapshots),  (http://nexus-private.hortonworks.com/nexus/content/groups/public, releases+snapshots),  (https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local, releases+snapshots)]].
2018/02/26 03:47:22 INFO    : org.eclipse.aether.resolution.DependencyResolutionException: Failed to collect dependencies at io.druid.extensions:druid-hdfs-storage:jar:0.10.1.2.6.5.0-129 -> org.apache.hadoop:hadoop-client:jar:2.7.3.2.6.5.0-129 -> org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-129 -> org.apache.hadoop:hadoop-auth:jar:2.7.3.2.6.5.0-129 -> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 -> net.minidev:json-smart:jar:2.3-SNAPSHOT
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:380) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	at io.tesla.aether.internal.DefaultTeslaAether.resolveArtifacts(DefaultTeslaAether.java:289) ~[tesla-aether-0.0.5.jar:0.0.5]
2018/02/26 03:47:22 INFO    : 	at io.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:350) [druid-services-0.10.1.2.6.5.0-129.jar:0.10.1.2.6.5.0-129]
2018/02/26 03:47:22 INFO    : 	at io.druid.cli.PullDependencies.run(PullDependencies.java:249) [druid-services-0.10.1.2.6.5.0-129.jar:0.10.1.2.6.5.0-129]
2018/02/26 03:47:22 INFO    : 	at io.druid.cli.Main.main(Main.java:108) [druid-services-0.10.1.2.6.5.0-129.jar:0.10.1.2.6.5.0-129]
2018/02/26 03:47:22 INFO    : Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to collect dependencies at io.druid.extensions:druid-hdfs-storage:jar:0.10.1.2.6.5.0-129 -> org.apache.hadoop:hadoop-client:jar:2.7.3.2.6.5.0-129 -> org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-129 -> org.apache.hadoop:hadoop-auth:jar:2.7.3.2.6.5.0-129 -> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 -> net.minidev:json-smart:jar:2.3-SNAPSHOT
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:292) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	... 4 more
2018/02/26 03:47:22 INFO    : Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for net.minidev:json-smart:jar:2.3-SNAPSHOT
2018/02/26 03:47:22 INFO    : 	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:335) ~[maven-aether-provider-3.1.1.jar:3.1.1]
2018/02/26 03:47:22 INFO    : 	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217) ~[maven-aether-provider-3.1.1.jar:3.1.1]
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:461) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342) ~[aether-impl-0.9.0.M2.jar:?]
2018/02/26 03:47:22 INFO    : 	... 4 more
2018/02/26 03:47:22 INFO    : Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact net.minidev:json-smart:pom:2.3-SNAPSHOT from/to  (https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local): Failed to transfer https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local/net/minidev/json-smart/2.3-SNAPSHOT/json-smart-2.3-SNAPSHOT.pom. Error code 409, Conflict