[SPARK-44477] CheckAnalysis uses error subclass as an error class

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.5.0, 4.0.0
• Component/s: SQL
• Assignee: Bruce Robbins
• Reporter: Bruce Robbins
• Created: 18/Jul/23 17:41
• Updated: 21/Jul/23 00:45
• Resolved: 21/Jul/23 00:45

Description

CheckAnalysis treats TYPE_CHECK_FAILURE_WITH_HINT as an error class, but it is instead an error subclass of DATATYPE_MISMATCH.

spark-sql (default)> select bitmap_count(12);
[INTERNAL_ERROR] Cannot find main error class 'TYPE_CHECK_FAILURE_WITH_HINT'
org.apache.spark.SparkException: [INTERNAL_ERROR] Cannot find main error class 'TYPE_CHECK_FAILURE_WITH_HINT'
at org.apache.spark.SparkException$.internalError(SparkException.scala:83)
at org.apache.spark.SparkException$.internalError(SparkException.scala:87)
at org.apache.spark.ErrorClassesJsonReader.$anonfun$getMessageTemplate$1(ErrorClassesJSONReader.scala:68)
at scala.collection.immutable.HashMap$HashMap1.getOrElse0(HashMap.scala:361)
at scala.collection.immutable.HashMap$HashTrieMap.getOrElse0(HashMap.scala:594)
at scala.collection.immutable.HashMap$HashTrieMap.getOrElse0(HashMap.scala:589)
at scala.collection.immutable.HashMap.getOrElse(HashMap.scala:73)

This issue only occurs when an expression uses TypeCheckResult.TypeCheckFailure to indicate input type check failure. TypeCheckResult.TypeCheckFailure appears to be deprecated in favor of TypeCheckResult.DataTypeMismatch, but recently two expressions were added that use TypeCheckResult.TypeCheckFailure: BitmapCount and BitmapOrAgg.

BitmapCount and BitmapOrAgg should probably be fixed to use TypeCheckResult.DataTypeMismatch. Regardless, the code in CheckAnalysis that handles TypeCheckResult.TypeCheckFailure should be corrected (or removed).