Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27710

ClassNotFoundException: $line196400984558.$read$ in OuterScopes

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.4.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None

      Description

      My colleague hit the following exception when using Spark in a Zeppelin notebook:

      java.lang.ClassNotFoundException: $line196400984558.$read$
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      	at java.lang.Class.forName0(Native Method)
      	at java.lang.Class.forName(Class.java:348)
      	at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
      	at org.apache.spark.sql.catalyst.encoders.OuterScopes$$anonfun$getOuterScope$1.apply(OuterScopes.scala:62)
      	at org.apache.spark.sql.catalyst.expressions.objects.NewInstance$$anonfun$10.apply(objects.scala:485)
      	at org.apache.spark.sql.catalyst.expressions.objects.NewInstance$$anonfun$10.apply(objects.scala:485)
      	at scala.Option.map(Option.scala:146)
      	at org.apache.spark.sql.catalyst.expressions.objects.NewInstance.doGenCode(objects.scala:485)
      	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:108)
      	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:105)
      	at scala.Option.getOrElse(Option.scala:121)
      	at org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:105)
      	at org.apache.spark.sql.catalyst.expressions.If.doGenCode(conditionalExpressions.scala:70)
      	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:108)
      	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:105)
      	at scala.Option.getOrElse(Option.scala:121)
      	at org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:105)
      	at org.apache.spark.sql.catalyst.expressions.objects.InvokeLike$$anonfun$3.apply(objects.scala:99)
      	at org.apache.spark.sql.catalyst.expressions.objects.InvokeLike$$anonfun$3.apply(objects.scala:98)
      	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
      	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
      	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
      	at scala.collection.AbstractTraversable.map(Traversable.scala:104)
      	at org.apache.spark.sql.catalyst.expressions.objects.InvokeLike$class.prepareArguments(objects.scala:98)
      	at org.apache.spark.sql.catalyst.expressions.objects.NewInstance.prepareArguments(objects.scala:431)
      	at org.apache.spark.sql.catalyst.expressions.objects.NewInstance.doGenCode(objects.scala:483)
      	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:108)
      	at org.apache.spark.sql.catalyst.expressions.Expression$$anonfun$genCode$2.apply(Expression.scala:105)
      	at scala.Option.getOrElse(Option.scala:121)
      	at org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:105)
      	at org.apache.spark.sql.execution.DeserializeToObjectExec.doConsume(objects.scala:84)
      	at org.apache.spark.sql.execution.CodegenSupport$class.constructDoConsumeFunction(WholeStageCodegenExec.scala:211)
      	at org.apache.spark.sql.execution.CodegenSupport$class.consume(WholeStageCodegenExec.scala:182)
      	at org.apache.spark.sql.execution.InputAdapter.consume(WholeStageCodegenExec.scala:357)
      	at org.apache.spark.sql.execution.InputAdapter.doProduce(WholeStageCodegenExec.scala:386)
      	at org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)
      	at org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)
      	at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
      	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
      	at org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)
      	at org.apache.spark.sql.execution.InputAdapter.produce(WholeStageCodegenExec.scala:357)
      	at org.apache.spark.sql.execution.DeserializeToObjectExec.doProduce(objects.scala:80)
      	at org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)
      	at org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)
      	at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
      	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
      	at org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)
      	at org.apache.spark.sql.execution.DeserializeToObjectExec.produce(objects.scala:68)
      	at org.apache.spark.sql.execution.MapElementsExec.doProduce(objects.scala:211)
      	at org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)
      	at org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)
      	at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
      	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
      	at org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)
      	at org.apache.spark.sql.execution.MapElementsExec.produce(objects.scala:200)
      	at org.apache.spark.sql.execution.SerializeFromObjectExec.doProduce(objects.scala:114)
      	at org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:90)
      	at org.apache.spark.sql.execution.CodegenSupport$$anonfun$produce$1.apply(WholeStageCodegenExec.scala:85)
      	at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
      	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
      	at org.apache.spark.sql.execution.CodegenSupport$class.produce(WholeStageCodegenExec.scala:85)
      	at org.apache.spark.sql.execution.SerializeFromObjectExec.produce(objects.scala:101)
      	at org.apache.spark.sql.execution.WholeStageCodegenExec.doCodeGen(WholeStageCodegenExec.scala:527)
      	at org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:581)
      	at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
      	at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
      	at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
      	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
      	at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
      	at org.apache.spark.sql.execution.SparkPlan.getByteArrayRdd(SparkPlan.scala:247)
      	at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:339)
      	at org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:38)
      	at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$collectFromPlan(Dataset.scala:3384)
      	at org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:2545)
      	at org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:2545)
      	at org.apache.spark.sql.Dataset$$anonfun$53.apply(Dataset.scala:3365)
      	at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)
      	at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
      	at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
      	at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3364)
      	at org.apache.spark.sql.Dataset.head(Dataset.scala:2545)
      [...]
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.zeppelin.spark.SparkZeppelinContext.showData(SparkZeppelinContext.java:108)
      	at org.apache.zeppelin.spark.SparkSqlInterpreter.interpret(SparkSqlInterpreter.java:135)
      	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:103)
      	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:632)
      	at org.apache.zeppelin.scheduler.Job.run(Job.java:188)
      	at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	[...]
      

      The ClassNotFoundException is occurring in Spark SQL's OuterScopes class at https://github.com/apache/spark/blob/v2.4.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/OuterScopes.scala#L62

      It looks like another Zeppelin user reported a similar problem at http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/zeppelin-spark-sql-ClassNotFoundException-line70280873551-read-td4556.html

      Our reproduction involves a REPL-defined case class named  $line196400984558.$read$$iw$$iw$OurCaseClassName appearing in a ObjectHashAggregate (with a newInstance(class $line196400984558.$read$$iw$$iw$OurCaseClassName) appearing in the plan).

      Unfortunately I don't have a standalone reproduction that I can share right now, but I wanted to report this issue for tracking / reference purposes.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              joshrosen Josh Rosen
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: