[SPARK-41735] Any SparkThrowable (with an error class) not in error-classes.json is masked in SQLExecution.withNewExecutionId and end-user will see "org.apache.spark.SparkException: [INTERNAL_ERROR]"

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Assignee: XiDuo You
• Reporter: Allison Portis
• Created: 28/Dec/22 01:47
• Updated: 30/Jan/23 08:07
• Resolved: 30/Jan/23 08:05

Description

This change here means that any seen error is passed to SparkThrowableHelper.getMessage(...). Any SparkThrowable with an error class (for example, if a connector uses the spark error format i.e. see ErrorClassesJsonReader) will be masked as 

org.apache.spark.SparkException: [INTERNAL_ERROR] Cannot find main error class 'SOME_ERROR_CLASS'

in SparkThrowableHelper.getMessage since errorReader.getMessageTemplate(errorClass) will fail for the error class not defined in error-classes.json.

 

Issue Links

• links to: [Github] Pull Request #39794 (ulysses-you)
• links to: [Github] Pull Request #39794 (ulysses-you)