[IMPALA-12580] Iceberg predicate filtering fails with UnsupportedOperationException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.3.0
• Fix Version/s: Impala 4.3.0
• Component/s: fe
• Labels: iceberg impala-iceberg
• Epic Color: ghx-label-8
• Assignee: Peter Rozsa
• Reporter: Peter Rozsa
• Created: 28/Nov/23 10:24
• Updated: 04/Jun/24 17:37
• Resolved: 01/Dec/23 08:13

Description

If a SELECT contains a predicate that is a candidate for propagation like "a = 10 and b = a", the optimizer replaces these predicates with a new one, where the id of the predicate is null.

There's a check in the Expr's hashCode implementation, if the id of the expression is null, we throw an UnsupportedOperationException.

The problem surfaced after IMPALA-12089, to fix this problem, proper cloning and id generation should be used for every expression. (https://github.com/apache/impala/blob/6e90dafcf4b8528061c5ae20933e112c355da834/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L2519)

To avoid this problem for Iceberg expression, a quick fix can be applied by replacing the collection of expressions from toSet() to toList()

 

Minimal reproducible example: 

create table i1(a int, b int, c int) partitioned by spec(a, b) stored as iceberg;
insert into i1(a, b, c) values (1, 1, 1);
select * from i1 where a = 1 and b = a and c = 1;

Issue Links

• is caused by: IMPALA-12089 Be able to skip pushing down a subset of the predicates