[SPARK-49200] Fix null type non-codegen ordering exception

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 4.0.0, 3.5.2
• Fix Version/s: 4.0.0, 3.5.3
• Component/s: SQL
• Labels: pull-request-available
• Assignee: XiDuo You
• Reporter: XiDuo You
• Created: 12/Aug/24 02:43
• Updated: 12/Aug/24 12:20
• Resolved: 12/Aug/24 10:11

Description

set spark.sql.codegen.factoryMode=NO_CODEGEN;
set spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.EliminateSorts;

select * from range(10) order by array(null);

org.apache.spark.SparkIllegalArgumentException: Type PhysicalNullType does not support ordered operations.
    at org.apache.spark.sql.errors.QueryExecutionErrors$.orderedOperationUnsupportedByDataTypeError(QueryExecutionErrors.scala:352)
    at org.apache.spark.sql.catalyst.types.PhysicalNullType.ordering(PhysicalDataType.scala:246)
    at org.apache.spark.sql.catalyst.types.PhysicalNullType.ordering(PhysicalDataType.scala:243)
    at org.apache.spark.sql.catalyst.types.PhysicalArrayType$$anon$1.<init>(PhysicalDataType.scala:283)
    at org.apache.spark.sql.catalyst.types.PhysicalArrayType.interpretedOrdering$lzycompute(PhysicalDataType.scala:281)
    at org.apache.spark.sql.catalyst.types.PhysicalArrayType.interpretedOrdering(PhysicalDataType.scala:281)
    at org.apache.spark.sql.catalyst.types.PhysicalArrayType.ordering(PhysicalDataType.scala:277)
    at org.apache.spark.sql.catalyst.expressions.InterpretedOrdering.compare(ordering.scala:67)
    at org.apache.spark.sql.catalyst.expressions.InterpretedOrdering.compare(ordering.scala:40)
    at org.apache.spark.sql.execution.UnsafeExternalRowSorter$RowComparator.compare(UnsafeExternalRowSorter.java:254)
    at org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorter$SortComparator.compare(UnsafeInMemorySorter.java:70)
    at org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorter$SortComparator.compare(UnsafeInMemorySorter.java:44)

Issue Links

• links to: GitHub Pull Request #47707