[SPARK-49480] NullPointerException from SparkThrowableHelper.isInternalError method

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0, 3.5.2
• Fix Version/s: 4.0.0, 3.5.3
• Component/s: Spark Core
• Labels: pull-request-available
• Assignee: Xi Chen
• Reporter: Xi Chen
• Created: 31/Aug/24 06:45
• Updated: 02/Sep/24 06:18
• Resolved: 02/Sep/24 05:59

Description

The SparkThrowableHelper.isInternalError method doesn't handle null input, and it could lead to NullPointerException.

Example stacktrace from our environment with Spark 3.5.1:

 Caused by: java.lang.NullPointerException: Cannot invoke "String.startsWith(String)" because "errorClass" is null
    at org.apache.spark.SparkThrowableHelper$.isInternalError(SparkThrowableHelper.scala:64)
    at org.apache.spark.SparkThrowableHelper.isInternalError(SparkThrowableHelper.scala)
    at org.apache.spark.SparkThrowable.isInternalError(SparkThrowable.java:50)
    at org.apache.spark.SparkException.isInternalError(SparkException.scala:27)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:688)
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:772)
    ... 30 more  

Issue Links

• links to: GitHub Pull Request #47946