[IMPALA-11346] Migrated partitioned Iceberg tables might return ERROR when WHERE condition is used on partition column

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0, Impala 4.1.0
• Fix Version/s: Impala 4.2.0, Impala 4.1.1
• Component/s: Backend
• Labels: impala-iceberg
• Epic Color: ghx-label-12
• Assignee: Zoltán Borók-Nagy
• Reporter: Zoltán Borók-Nagy
• Created: 09/Jun/22 13:36
• Updated: 26/Aug/22 00:07
• Resolved: 12/Jul/22 13:44

Description

[localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where p_bool=false;
Fetched 0 row(s) in 0.11s

[localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where p_bool=true;
ERROR: Unable to find SchemaNode for path 'functional_parquet.iceberg_alltypes_part.p_bool' in the schema of file 'hdfs://localhost:20500/test-warehouse/iceberg_test/hadoop_catalog/ice/iceberg_alltypes_part/p_bool=true/p_int=1/p_bigint=11/p_float=1.1/p_double=2.222/p_decimal=123.321/p_date=2022-02-22/p_string=impala/000000_0'.

[localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where i=3;
Fetched 0 row(s) in 0.12s

[localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where i=1;
+---+--------+-------+----------+---------------+----------+-----------+------------+----------+
| i | p_bool | p_int | p_bigint | p_float       | p_double | p_decimal | p_date     | p_string |
+---+--------+-------+----------+---------------+----------+-----------+------------+----------+
| 1 | true   | 1     | 11       | 1.10000002384 | 2.222    | 123.321   | 2022-02-22 | impala   |
+---+--------+-------+----------+---------------+----------+-----------+------------+----------+
Fetched 1 row(s) in 0.12s

[localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where p_int=1;
ERROR: Unable to find SchemaNode for path 'functional_parquet.iceberg_alltypes_part.p_int' in the schema of file 'hdfs://localhost:20500/test-warehouse/iceberg_test/hadoop_catalog/ice/iceberg_alltypes_part/p_bool=true/p_int=1/p_bigint=11/p_float=1.1/p_double=2.222/p_decimal=123.321/p_date=2022-02-22/p_string=impala/000000_0'.

[localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where p_int=3;
Fetched 0 row(s) in 0.11s

So we don't get incorrect results at least, but getting errors on partition column values that are existing.

It seems like it works well with ORC.