[IMPALA-13002] Iceberg V2 tables with Avro delete files aren't read properly

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.4.0
• Component/s: Backend
• Labels: impala-iceberg
• Epic Color: ghx-label-5
• Assignee: Zoltán Borók-Nagy
• Reporter: Zoltán Borók-Nagy
• Created: 15/Apr/24 06:25
• Updated: 4 days ago 20:41
• Resolved: 23/Apr/24 09:16

Description

If the Iceberg table has Avro delete files (e.g. by setting 'write.delete.format.default'='avro') then Impala won't be able to read the contents of the delete files properly.

Therefore delete records won't be applied to the Iceberg table during reads. The new Iceberg V2 operator will raise errors in this case, but if it is disabled via 'set disable_optimized_iceberg_v2_read=true' then Impala won't be able to detect invalid delete records.

Issue Links

• causes: IMPALA-13046 Rework Iceberg mixed format delete test for Hive optimization