Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4825

CTAS fails to resolve when created using saveAsTable

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.2.0
    • 1.2.1
    • SQL
    • None

    Description

      While writing a test for a different issue, I found that saveAsTable seems to be broken:

        test("save join to table") {
          val testData = sparkContext.parallelize(1 to 10).map(i => TestData(i, i.toString))
          sql("CREATE TABLE test1 (key INT, value STRING)")
          testData.insertInto("test1")
          sql("CREATE TABLE test2 (key INT, value STRING)")
          testData.insertInto("test2")
          testData.insertInto("test2")
          sql("SELECT COUNT(a.value) FROM test1 a JOIN test2 b ON a.key = b.key").saveAsTable("test")
          checkAnswer(
            table("test"),
            sql("SELECT COUNT(a.value) FROM test1 a JOIN test2 b ON a.key = b.key").collect().toSeq)
        }
      ​
          sql("SELECT COUNT(a.value) FROM test1 a JOIN test2 b ON a.key = b.key").saveAsTable("test")
      org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Unresolved plan found, tree:
      'CreateTableAsSelect None, test, false, None
       Aggregate [], [COUNT(value#336) AS _c0#334L]
        Join Inner, Some((key#335 = key#339))
         MetastoreRelation default, test1, Some(a)
         MetastoreRelation default, test2, Some(b)
      

      Attachments

        Activity

          People

            chenghao Cheng Hao
            marmbrus Michael Armbrust
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: