Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17714

ClassCircularityError is thrown when using org.apache.spark.util.Utils.classForName 

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.1, 2.2.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      This jira is a follow up to SPARK-15857 .

      Task invokes CallerContext. SetCurrentContext() to set its callerContext to HDFS. In SetCurrentContext(), it tries looking for class org.apache.hadoop.ipc.CallerContext by using org.apache.spark.util.Utils.classForName. This causes ClassCircularityError to be thrown when running ReplSuite in master Maven builds (The same tests pass in the SBT build). A hotfix SPARK-17710 has been made by using Class.forName instead, but it needs further investigation.

      Error:
      https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-maven-hadoop-2.3/2000/testReport/junit/org.apache.spark.repl/ReplSuite/simple_foreach_with_accumulator/

      scala> accum: org.apache.spark.util.LongAccumulator = LongAccumulator(id: 0, name: None, value: 0)
      scala> org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.ClassCircularityError: io/netty/util/internal/_matchers_/org/apache/spark/network/protocol/MessageMatcher
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at io.netty.util.internal.JavassistTypeParameterMatcherGenerator.generate(JavassistTypeParameterMatcherGenerator.java:62)
      at io.netty.util.internal.JavassistTypeParameterMatcherGenerator.generate(JavassistTypeParameterMatcherGenerator.java:54)
      at io.netty.util.internal.TypeParameterMatcher.get(TypeParameterMatcher.java:42)
      at io.netty.util.internal.TypeParameterMatcher.find(TypeParameterMatcher.java:78)
      at io.netty.handler.codec.MessageToMessageEncoder.<init>(MessageToMessageEncoder.java:59)
      at org.apache.spark.network.protocol.MessageEncoder.<init>(MessageEncoder.java:34)
      at org.apache.spark.network.TransportContext.<init>(TransportContext.java:78)
      at org.apache.spark.rpc.netty.NettyRpcEnv.downloadClient(NettyRpcEnv.scala:354)
      at org.apache.spark.rpc.netty.NettyRpcEnv.openChannel(NettyRpcEnv.scala:324)
      at org.apache.spark.repl.ExecutorClassLoader.org$apache$spark$repl$ExecutorClassLoader$$getClassFileInputStreamFromSparkRPC(ExecutorClassLoader.scala:90)
      at org.apache.spark.repl.ExecutorClassLoader$$anonfun$1.apply(ExecutorClassLoader.scala:57)
      at org.apache.spark.repl.ExecutorClassLoader$$anonfun$1.apply(ExecutorClassLoader.scala:57)
      at org.apache.spark.repl.ExecutorClassLoader.findClassLocally(ExecutorClassLoader.scala:162)
      at org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:80)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at io.netty.util.internal.JavassistTypeParameterMatcherGenerator.generate(JavassistTypeParameterMatcherGenerator.java:62)
      at io.netty.util.internal.JavassistTypeParameterMatcherGenerator.generate(JavassistTypeParameterMatcherGenerator.java:54)
      at io.netty.util.internal.TypeParameterMatcher.get(TypeParameterMatcher.java:42)
      at io.netty.util.internal.TypeParameterMatcher.find(TypeParameterMatcher.java:78)
      at io.netty.handler.codec.MessageToMessageEncoder.<init>(MessageToMessageEncoder.java:59)
      at org.apache.spark.network.protocol.MessageEncoder.<init>(MessageEncoder.java:34)
      at org.apache.spark.network.TransportContext.<init>(TransportContext.java:78)
      at org.apache.spark.rpc.netty.NettyRpcEnv.downloadClient(NettyRpcEnv.scala:354)
      at org.apache.spark.rpc.netty.NettyRpcEnv.openChannel(NettyRpcEnv.scala:324)
      at org.apache.spark.repl.ExecutorClassLoader.org$apache$spark$repl$ExecutorClassLoader$$getClassFileInputStreamFromSparkRPC(ExecutorClassLoader.scala:90)
      at org.apache.spark.repl.ExecutorClassLoader$$anonfun$1.apply(ExecutorClassLoader.scala:57)
      at org.apache.spark.repl.ExecutorClassLoader$$anonfun$1.apply(ExecutorClassLoader.scala:57)
      at org.apache.spark.repl.ExecutorClassLoader.findClassLocally(ExecutorClassLoader.scala:162)
      at org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:80)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at org.apache.spark.util.Utils$.classForName(Utils.scala:225)
      at org.apache.spark.util.CallerContext.setCurrentContext(Utils.scala:2492)
      at org.apache.spark.scheduler.Task.run(Task.scala:96)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      Driver stacktrace:
      at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1433)
      at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1421)
      at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1420)
      at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
      at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
      at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1420)
      at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
      at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
      at scala.Option.foreach(Option.scala:257)
      at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:802)
      at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1648)
      at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1603)
      at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1592)
      at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
      at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)
      at org.apache.spark.SparkContext.runJob(SparkContext.scala:1901)
      at org.apache.spark.SparkContext.runJob(SparkContext.scala:1914)
      at org.apache.spark.SparkContext.runJob(SparkContext.scala:1927)
      at org.apache.spark.SparkContext.runJob(SparkContext.scala:1941)
      at org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:894)
      at org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:892)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
      at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
      at org.apache.spark.rdd.RDD.foreach(RDD.scala:892)
      ... 94 elided
      Caused by: java.lang.ClassCircularityError: io/netty/util/internal/_matchers_/org/apache/spark/network/protocol/MessageMatcher
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at io.netty.util.internal.JavassistTypeParameterMatcherGenerator.generate(JavassistTypeParameterMatcherGenerator.java:62)
      at io.netty.util.internal.JavassistTypeParameterMatcherGenerator.generate(JavassistTypeParameterMatcherGenerator.java:54)
      at io.netty.util.internal.TypeParameterMatcher.get(TypeParameterMatcher.java:42)
      at io.netty.util.internal.TypeParameterMatcher.find(TypeParameterMatcher.java:78)
      at io.netty.handler.codec.MessageToMessageEncoder.<init>(MessageToMessageEncoder.java:59)
      at org.apache.spark.network.protocol.MessageEncoder.<init>(MessageEncoder.java:34)
      at org.apache.spark.network.TransportContext.<init>(TransportContext.java:78)
      at org.apache.spark.rpc.netty.NettyRpcEnv.downloadClient(NettyRpcEnv.scala:354)
      at org.apache.spark.rpc.netty.NettyRpcEnv.openChannel(NettyRpcEnv.scala:324)
      at org.apache.spark.repl.ExecutorClassLoader.org$apache$spark$repl$ExecutorClassLoader$$getClassFileInputStreamFromSparkRPC(ExecutorClassLoader.scala:90)
      at org.apache.spark.repl.ExecutorClassLoader$$anonfun$1.apply(ExecutorClassLoader.scala:57)
      at org.apache.spark.repl.ExecutorClassLoader$$anonfun$1.apply(ExecutorClassLoader.scala:57)
      at org.apache.spark.repl.ExecutorClassLoader.findClassLocally(ExecutorClassLoader.scala:162)
      at org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:80)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at io.netty.util.internal.JavassistTypeParameterMatcherGenerator.generate(JavassistTypeParameterMatcherGenerator.java:62)
      at io.netty.util.internal.JavassistTypeParameterMatcherGenerator.generate(JavassistTypeParameterMatcherGenerator.java:54)
      at io.netty.util.internal.TypeParameterMatcher.get(TypeParameterMatcher.java:42)
      at io.netty.util.internal.TypeParameterMatcher.find(TypeParameterMatcher.java:78)
      at io.netty.handler.codec.MessageToMessageEncoder.<init>(MessageToMessageEncoder.java:59)
      at org.apache.spark.network.protocol.MessageEncoder.<init>(MessageEncoder.java:34)
      at org.apache.spark.network.TransportContext.<init>(TransportContext.java:78)
      at org.apache.spark.rpc.netty.NettyRpcEnv.downloadClient(NettyRpcEnv.scala:354)
      at org.apache.spark.rpc.netty.NettyRpcEnv.openChannel(NettyRpcEnv.scala:324)
      at org.apache.spark.repl.ExecutorClassLoader.org$apache$spark$repl$ExecutorClassLoader$$getClassFileInputStreamFromSparkRPC(ExecutorClassLoader.scala:90)
      at org.apache.spark.repl.ExecutorClassLoader$$anonfun$1.apply(ExecutorClassLoader.scala:57)
      at org.apache.spark.repl.ExecutorClassLoader$$anonfun$1.apply(ExecutorClassLoader.scala:57)
      at org.apache.spark.repl.ExecutorClassLoader.findClassLocally(ExecutorClassLoader.scala:162)
      at org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:80)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at org.apache.spark.util.Utils$.classForName(Utils.scala:225)
      at org.apache.spark.util.CallerContext.setCurrentContext(Utils.scala:2492)
      at org.apache.spark.scheduler.Task.run(Task.scala:96)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      scala> res1: Long = 0
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zsxwing Shixiong Zhu
                Reporter:
                WeiqingYang Weiqing Yang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: