[IMPALA-10569] Impala should determine Iceberg data file format from Iceberg metadata

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.3.0
• Component/s: Frontend
• Labels: impala-iceberg
• Epic Color: ghx-label-7
• Assignee: Noemi Pap-Takacs
• Reporter: Zoltán Borók-Nagy
• Created: 09/Mar/21 13:47
• Updated: 12/Dec/22 14:55
• Resolved: 12/Dec/22 14:55

Description

When Impala creates an Iceberg table it sets HMS table property 'iceberg.file_format' to indicate the underlying data file format.

However, when the table was created by Hive or Spark, we don't have this property and Impala assumes that the data file format is PARQUET. This assumption is just a wild guess, and when it's wrong Impala raises an error during query execution.

Instead of only checking the table property, Impala could also try to determine the file format based on Iceberg metadata.

Issue Links

• is fixed by: IMPALA-11158 Add support for Iceberg tables with AVRO data files