[SPARK-48286] Analyze 'exists' default expression instead of 'current' default expression in structField to v2 column conversion

• Type: Bug
• Status: Open
• Priority: Trivial
• Resolution: Unresolved
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Spark Core
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Uros Stankovic
• Created: 15/May/24 09:13
• Updated: 5 days ago 18:28

Description

org.apache.spark.sql.catalyst.util.ResolveDefaultColumns#analyze method accepts 3 parameter

1) Field to analyze
2) Statement type - String

3) Metadata key - CURRENT_DEFAULT or EXISTS_DEFAULT

Method org.apache.spark.sql.connector.catalog.CatalogV2Util#structFieldToV2Column
pass fieldToAnalyze and EXISTS_DEFAULT as second parameter, so it is not metadata key, instead of that, it is statement type, so bad expression is analyzed.

Issue Links

• links to: GitHub Pull Request #46594