[SPARK-49350] FoldablePropagation rule and ConstantFolding rule leads to wrong aggregated result

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 3.2.1
• Component/s: SQL

Language

scala

• Assignee: Unassigned
• Reporter: GANHONGNAN
• Created: 22/Aug/24 05:44
• Updated: 27/Aug/24 14:41
• Resolved: 27/Aug/24 14:41

Description

SELECT  cast(-1 AS BIGINT) AS ele1
FROM    (            
       SELECT  array(1, 5, 3, 123, 255, 546, 64, 23) AS t        
   ) LATERAL VIEW explode(t) tmp AS ele
WHERE   ele=-1 

This query returns an empty result. However, the following query returns 1.  This result seems wrong.

SELECT  count(DISTINCT ele1)
FROM    (            
    SELECT  cast(-1 as bigint) as ele1            
    FROM    (                        
        SELECT  array(1, 5, 3, 123, 255, 546, 64, 23) AS t                       
        ) LATERAL VIEW explode(t) tmp AS ele            
    WHERE   ele = -1        
) 

By plan change log, I find that it is FoldablePropagation rule and ConstantFolding rule that optimize Aggregate expression to `Aggregat [cast(count(distinct -1) as string) AS count(DISTINCT ele)#7 ]`.

 

Is this result right?  Does it need to be fixed? 

 

Issue Links

• duplicates: SPARK-49000 Aggregation with DISTINCT gives wrong results when dealing with literals