[SPARK-47262] Assign classes to Parquet converter errors

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available starter
• Assignee: Vladan Vasić
• Reporter: Max Gekk
• Shepherd: Max Gekk
• Created: 03/Mar/24 13:24
• Updated: 22/Aug/24 16:16
• Resolved: 22/Aug/24 16:15

Description

Choose a proper name for the error class _LEGACY_ERROR_TEMP_22[38-40] defined in core/src/main/resources/error/error-classes.json. The name should be short but complete (look at the example in error-classes.json).

Add a test which triggers the error from user code if such test still doesn't exist. Check exception fields by using checkError(). The last function checks valuable error fields only, and avoids dependencies from error text message. In this way, tech editors can modify error format in error-classes.json, and don't worry of Spark's internal tests. Migrate other tests that might trigger the error onto checkError().

If you cannot reproduce the error from user space (using SQL query), replace the error by an internal error, see SparkException.internalError().

Improve the error message format in error-classes.json if the current is not clear. Propose a solution to users how to avoid and fix such kind of errors.

Please, look at the PR below as examples:

• https://github.com/apache/spark/pull/38685
• https://github.com/apache/spark/pull/38656
• https://github.com/apache/spark/pull/38490

Issue Links

• is a clone of: SPARK-47261 Assign classes to Parquet type errors
• is cloned by: SPARK-47263 Assign classes to DEFAULT value errors
• links to: GitHub Pull Request #47802