[SPARK-44420] CheckOverflowInTableInsert does not handle cases where the cast was moved by the optimizer

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.4.0
• Component/s: Optimizer
• Assignee: Unassigned
• Reporter: Jan-Ole Sasse
• Created: 14/Jul/23 07:10
• Updated: 14/Jul/23 07:10

Description

CheckOverflowInTableInsert is supposed to rewrite error messages that come from overflow exceptions on casts that are implicitly inserted for INSERTs to match the output schema. This does not work in all cases, for example when the cast is moved by the optimizer
 

CREATE TABLE source(value BIGINT) USING parquet
INSERT INTO source values (${Long.MaxValue.toString})
CREATE TABLE target(value INT) USING parquet
INSERT INTO target SELECT CASE WHEN value < 0 THEN 0 ELSE value END FROM source

In this case, the expression will no nothing because it only becomes effective if the direct child is a cast (matching code)