[SPARK-34796] Codegen compilation error for query with LIMIT operator and without AQE

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.1.0, 3.1.1, 3.2.0
• Fix Version/s: 3.1.2, 3.2.0
• Component/s: SQL
• Assignee: Cheng Su
• Reporter: Cheng Su
• Created: 19/Mar/21 00:46
• Updated: 12/Dec/22 18:10
• Resolved: 20/Mar/21 02:22

Description

Example (reproduced in unit test):

 

  test("failed limit query") {
    withTable("left_table", "empty_right_table", "output_table") {
      spark.range(5).toDF("k").write.saveAsTable("left_table")
      spark.range(0).toDF("k").write.saveAsTable("empty_right_table")
         
      withSQLConf(SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "false") {
        spark.sql("CREATE TABLE output_table (k INT) USING parquet")
        spark.sql(
          s"""
             |INSERT INTO TABLE output_table
             |SELECT t1.k FROM left_table t1
             |JOIN empty_right_table t2
             |ON t1.k = t2.k
             |LIMIT 3
             |""".stripMargin)
      }
    }
  }

Result:

 

https://gist.github.com/c21/ea760c75b546d903247582be656d9d66

Issue Links

• links to: [Github] Pull Request #31892 (c21)
• links to: [Github] Pull Request #31911 (c21)