[SPARK-40429] Only set KeyGroupedPartitioning when the referenced column is in the output

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3.0, 3.4.0
• Fix Version/s: 3.3.1, 3.4.0
• Component/s: SQL
• Assignee: Huaxin Gao
• Reporter: Huaxin Gao
• Created: 14/Sep/22 20:19
• Updated: 28/Sep/22 03:32
• Resolved: 15/Sep/22 06:07

Description

      sql(s"CREATE TABLE $tbl (id bigint, data string) PARTITIONED BY (id)")
      sql(s"INSERT INTO $tbl VALUES (1, 'a'), (2, 'b'), (3, 'c')")
      checkAnswer(
        spark.table(tbl).select("index", "_partition"),
        Seq(Row(0, "3"), Row(0, "2"), Row(0, "1"))
      )

failed with
ScalaTestFailureLocation: org.apache.spark.sql.QueryTest at (QueryTest.scala:226)
org.scalatest.exceptions.TestFailedException: AttributeSet(id#994L) was not empty The optimized logical plan has missing inputs:
RelationV2index#998, _partition#999 testcat.t

Issue Links

• links to: [Github] Pull Request #37886 (huaxingao)
• links to: [Github] Pull Request #37901 (huaxingao)
• links to: [Github] Pull Request #37901 (huaxingao)