Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17324

Don't relocate org.bouncycastle in shaded client jars

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.1, 3.4.0
    • Component/s: None

      Description

      When downstream apps depend on hadoop-client-api, hadoop-client-runtime and hadoop-client-minicluster, it seems the MiniYARNCluster could have issue because org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException is not in any of the above jars.

      Error:  Caused by: sbt.ForkMain$ForkError: java.lang.ClassNotFoundException: org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
      Error:  	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
      Error:  	at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
      Error:  	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
      Error:  	at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
      Error:  	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:862)
      Error:  	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      Error:  	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1296)
      Error:  	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:339)
      Error:  	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      Error:  	at org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:353)
      Error:  	at org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:127)
      Error:  	at org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:488)
      Error:  	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      Error:  	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
      Error:  	at org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:321)
      Error:  	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      Error:  	at org.apache.spark.deploy.yarn.BaseYarnClusterSuite.beforeAll(BaseYarnClusterSuite.scala:94)
      Error:  	at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
      Error:  	at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
      Error:  	at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
      Error:  	at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
      Error:  	at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
      Error:  	at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
      Error:  	at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
      Error:  	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      Error:  	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      Error:  	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      Error:  	at java.lang.Thread.run(Thread.java:748)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                csun Chao Sun
                Reporter:
                csun Chao Sun
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 50m
                  4h 50m