[SPARK-46394] spark.catalog.listDatabases() fails if containing schemas with special characters when spark.sql.legacy.keepCommandOutputSchema set to true

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Xinyi Yu
• Reporter: Xinyi Yu
• Created: 13/Dec/23 19:14
• Updated: 14/Dec/23 06:19
• Resolved: 14/Dec/23 06:19

Description

When the SQL conf spark.sql.legacy.keepCommandOutputSchema is set to true:
Before:
// support there is a xyyu-db-with-hyphen schema in the catalog
spark.catalog.listDatabases()

[INVALID_IDENTIFIER] The identifier xyyu-db-with-hyphen is invalid. Please, consider quoting it with back-quotes

Issue Links

• links to: GitHub Pull Request #44328