[IMPALA-10150] Write Iceberg Tables in Parquet format

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Labels: impala-iceberg
• Epic Link: Iceberg support in Impala
• Epic Color: ghx-label-4
• Assignee: Zoltán Borók-Nagy
• Reporter: Zoltán Borók-Nagy
• Created: 05/Sep/20 10:41
• Updated: 19/Jan/23 10:38
• Resolved: 19/Jan/23 10:38

Description

Impala should be able to INSERT and INSERT OVERWRITE data into Iceberg tables that use the Parquet file format.

DELETEs and UPDATEs are out-of-scope of this task.

If possible, use the existing parquet writers of Impala. Only use the AppendFiles API of Iceberg to update the table snapshot.

Sub-Tasks

• Do a best-effort cleanup on failed INSERTs into Iceberg tables: Open
• Implement INSERT INTO for non-partitioned Iceberg tables (Parquet): Resolved
• Implement CREATE TABLE AS SELECT for Iceberg tables: Resolved
• Implement INSERT OVERWRITE for Iceberg tables: Resolved
• INSERT INTO identity-partitioned Iceberg tables: Resolved
• INSERT INTO Iceberg tables with partition transforms: Resolved
• Implement TRUNCATE for Iceberg tables: Resolved