Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-41144

UnresolvedHint should not cause query failure

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete CommentsDelete
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.3.2, 3.4.0
    • SQL
    • None

    Description

       

      CREATE TABLE t1(c1 bigint) USING PARQUET;
      CREATE TABLE t2(c2 bigint) USING PARQUET;
      SELECT /*+ hash(t2) */ * FROM t1 join t2 on c1 = c2;

       

       

      failed with msg:

      org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to exprId on unresolved object
        at org.apache.spark.sql.catalyst.analysis.UnresolvedAttribute.exprId(unresolved.scala:147)
        at org.apache.spark.sql.catalyst.analysis.Analyzer$AddMetadataColumns$.$anonfun$hasMetadataCol$4(Analyzer.scala:1005)
        at org.apache.spark.sql.catalyst.analysis.Analyzer$AddMetadataColumns$.$anonfun$hasMetadataCol$4$adapted(Analyzer.scala:1005)
        at scala.collection.Iterator.exists(Iterator.scala:969)
        at scala.collection.Iterator.exists$(Iterator.scala:967)
        at scala.collection.AbstractIterator.exists(Iterator.scala:1431)
        at scala.collection.IterableLike.exists(IterableLike.scala:79)
        at scala.collection.IterableLike.exists$(IterableLike.scala:78)
        at scala.collection.AbstractIterable.exists(Iterable.scala:56)
        at org.apache.spark.sql.catalyst.analysis.Analyzer$AddMetadataColumns$.$anonfun$hasMetadataCol$3(Analyzer.scala:1005)
        at org.apache.spark.sql.catalyst.analysis.Analyzer$AddMetadataColumns$.$anonfun$hasMetadataCol$3$adapted(Analyzer.scala:1005) 

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ulysses XiDuo You Assign to me
            ulysses XiDuo You
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment