Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31773

getting the Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:56)

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 2.2.0
    • Fix Version/s: None
    • Component/s: Spark Core
    • Labels:
      None
    • Environment:

      spark 2.2

      Description

      Actually I am loading the excel which has some 90 columns and the some columns name contains special character as well like @ % -> . etc etc so while I am doing one use case like :

      sourceDataSet.select(columnSeq).except(targetDataset.select(columnSeq)));

      this is working fine but as soon as I am running 

      sourceDataSet.select(columnSeq).except(targetDataset.select(columnSeq)).count()

      it is failing with error like :

      org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree:

      Exchange SinglePartition

      +- *HashAggregate(keys=[], functions=[partial_count(1)], output=count#26596L)

         +- *HashAggregate(keys=columns name 

       

       

      Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, tree:column namet#14050

              at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:56)

              at org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:88)

              at org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1.applyOrElse(BoundAttribute.scala:87)

              at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$2.apply(TreeNode.scala:267)

              at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$2.apply(TreeNode.scala:267)

              at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:70)

              at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:266)

              at org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:256)

              at org.apache.spark.sql.catalyst.expressions.BindReferences$.bindReference(BoundAttribute.scala:87)

              at org.apache.spark.sql.execution.aggregate.HashAggregateExec$$anonfun$40.apply(HashAggregateExec.scala:703)

              at org.apache.spark.sql.execution.aggregate.HashAggregateExec$$anonfun$40.apply(HashAggregateExec.scala:703)

              at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)

              at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)

              at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)

              at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)

              at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)

              at scala.collection.immutable.Stream$$anonfun$map$1.apply(Stream.scala:418)

              at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1233)

              at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1223)

              at scala.collection.immutable.Stream.foreach(Stream.scala:595)

              at scala.collection.TraversableOnce$class.count(TraversableOnce.scala:115)

              at scala.collection.AbstractTraversable.count(Traversable.scala:104)

              at org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection$.createCode(GenerateUnsafeProjection.scala:312)

              at org.apache.spark.sql.execution.aggregate.HashAggregateExec.doConsumeWithKeys(HashAggregateExec.scala:702)

              at org.apache.spark.sql.execution.aggregate.HashAggregateExec.doConsume(HashAggregateExec.scala:156)

              at org.apache.spark.sql.execution.CodegenSupport$class.consume(WholeStageCodegenExec.scala:155)

              at org.apache.spark.sql.execution.ProjectExec.consume(basicPhysicalOperators.scala:36)

       

       

       

       

      Caused by: java.lang.RuntimeException: Couldn't find here one name of column following with

        at scala.sys.package$.error(package.scala:27)

              at org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1$$anonfun$applyOrElse$1.apply(BoundAttribute.scala:94)

              at org.apache.spark.sql.catalyst.expressions.BindReferences$$anonfun$bindReference$1$$anonfun$applyOrElse$1.apply(BoundAttribute.scala:88)

              at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:52)

       

       

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                pankaj24 Pankaj Tiwari
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: