Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15265

Exclude json-smart explicitly in hadoop-auth avoid being pulled in transitively

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.1.0
    • None
    • None
    • Reviewed

    Description

      this is an extension of - https://issues.apache.org/jira/browse/HADOOP-14903

      We need to exclude the dependency explicitly in hadoop-auth pom.xml and add the correct version so that it is not being pulled transitively. 

      In Druid we use, https://github.com/tesla/tesla-aether/blob/master/src/main/java/io/tesla/aether/TeslaAether.java to fetch dependencies transitively, which is still pulling in wrong version of json-smart jar.

      org.apache.hadoop:hadoop-auth:jar:2.7.3.2.6.5.0-129 -> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 -> net.minidev:json-smart:jar:2.3-SNAPSHOT

       

      Full Stack trace 

       2018/02/26 03:47:22 INFO    : 2018-02-26T03:47:22,878 ERROR [main] io.druid.cli.PullDependencies - Unable to resolve artifacts for [io.druid.extensions:druid-hdfs-storage:jar:0.10.1.2.6.5.0-129 (runtime) -> [] < [ (https://repo1.maven.org/maven2/, releases+snapshots),  (http://nexus-private.hortonworks.com/nexus/content/groups/public, releases+snapshots),  (http://nexus-private.hortonworks.com/nexus/content/groups/public, releases+snapshots),  (http://nexus-private.hortonworks.com/nexus/content/groups/public, releases+snapshots),  (https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local, releases+snapshots)]].
      2018/02/26 03:47:22 INFO    : org.eclipse.aether.resolution.DependencyResolutionException: Failed to collect dependencies at io.druid.extensions:druid-hdfs-storage:jar:0.10.1.2.6.5.0-129 -> org.apache.hadoop:hadoop-client:jar:2.7.3.2.6.5.0-129 -> org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-129 -> org.apache.hadoop:hadoop-auth:jar:2.7.3.2.6.5.0-129 -> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 -> net.minidev:json-smart:jar:2.3-SNAPSHOT
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:380) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	at io.tesla.aether.internal.DefaultTeslaAether.resolveArtifacts(DefaultTeslaAether.java:289) ~[tesla-aether-0.0.5.jar:0.0.5]
      2018/02/26 03:47:22 INFO    : 	at io.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:350) [druid-services-0.10.1.2.6.5.0-129.jar:0.10.1.2.6.5.0-129]
      2018/02/26 03:47:22 INFO    : 	at io.druid.cli.PullDependencies.run(PullDependencies.java:249) [druid-services-0.10.1.2.6.5.0-129.jar:0.10.1.2.6.5.0-129]
      2018/02/26 03:47:22 INFO    : 	at io.druid.cli.Main.main(Main.java:108) [druid-services-0.10.1.2.6.5.0-129.jar:0.10.1.2.6.5.0-129]
      2018/02/26 03:47:22 INFO    : Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to collect dependencies at io.druid.extensions:druid-hdfs-storage:jar:0.10.1.2.6.5.0-129 -> org.apache.hadoop:hadoop-client:jar:2.7.3.2.6.5.0-129 -> org.apache.hadoop:hadoop-common:jar:2.7.3.2.6.5.0-129 -> org.apache.hadoop:hadoop-auth:jar:2.7.3.2.6.5.0-129 -> com.nimbusds:nimbus-jose-jwt:jar:4.41.1 -> net.minidev:json-smart:jar:2.3-SNAPSHOT
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:292) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	... 4 more
      2018/02/26 03:47:22 INFO    : Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for net.minidev:json-smart:jar:2.3-SNAPSHOT
      2018/02/26 03:47:22 INFO    : 	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:335) ~[maven-aether-provider-3.1.1.jar:3.1.1]
      2018/02/26 03:47:22 INFO    : 	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217) ~[maven-aether-provider-3.1.1.jar:3.1.1]
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:461) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342) ~[aether-impl-0.9.0.M2.jar:?]
      2018/02/26 03:47:22 INFO    : 	... 4 more
      2018/02/26 03:47:22 INFO    : Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact net.minidev:json-smart:pom:2.3-SNAPSHOT from/to  (https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local): Failed to transfer https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local/net/minidev/json-smart/2.3-SNAPSHOT/json-smart-2.3-SNAPSHOT.pom. Error code 409, Conflict
      
      

       

      Attachments

        1. HADOOP-15265.2.patch
          1 kB
          Nishant Bangarwa
        2. HADOOP-15265.patch
          1 kB
          Nishant Bangarwa

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            nishantbangarwa Nishant Bangarwa
            nishantbangarwa Nishant Bangarwa
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment