[SPARK-41931] Improve UNSUPPORTED_DATA_TYPE message for complex types

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.5.0
• Component/s: Spark Core
• Assignee: Max Gekk
• Reporter: Serge Rielau
• Shepherd: Max Gekk
• Created: 07/Jan/23 03:43
• Updated: 01/Feb/23 05:10
• Resolved: 27/Jan/23 15:13

Description

spark-sql> SELECT CAST(array(1, 2, 3) AS ARRAY);

[UNSUPPORTED_DATATYPE] Unsupported data type "ARRAY"(line 1, pos 30)

== SQL ==

SELECT CAST(array(1, 2, 3) AS ARRAY)

------------------------------^^^

This error message is confusing. We support ARRAY. We just require it to be typed.
We should have an error like:
[INCOMPLETE_TYPE_DEFINITION.ARRAY] The definition of type `ARRAY` is incomplete. You must provide an element type. For example: `ARRAY\<INT\>`.
Similarly for STRUCT and MAP.  

Issue Links

• links to: [Github] Pull Request #39711 (RunyaoChen)
• links to: [Github] Pull Request #39836 (itholic)