[SPARK-34556] Checking duplicate static partition columns doesn't respect case sensitive conf

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.7, 3.0.2, 3.1.1
• Fix Version/s: 3.0.3, 3.1.2
• Component/s: SQL
• Assignee: Shixiong Zhu
• Reporter: Shixiong Zhu
• Created: 27/Feb/21 01:10
• Updated: 12/Dec/22 18:10
• Resolved: 03/Mar/21 02:36

Description

When parsing the partition spec, Spark will call `org.apache.spark.sql.catalyst.parser.ParserUtils.checkDuplicateKeys` to check if there are duplicate partition column names in the list. But this method is always case sensitive and doesn't discover duplicate partition column names when using different cases.

Issue Links

• links to: [Github] Pull Request #31669 (zsxwing)
• links to: [Github] Pull Request #31669 (zsxwing)
• links to: [Github] Pull Request #31714 (zsxwing)
• links to: [Github] Pull Request #31714 (zsxwing)