[IMPALA-10358] Correct Iceberg type mappings

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Labels: impala-iceberg
• Epic Link: Iceberg support in Impala
• Epic Color: ghx-label-9
• Assignee: Zoltán Borók-Nagy
• Reporter: Zoltán Borók-Nagy
• Created: 25/Nov/20 18:23
• Updated: 17/Dec/20 12:00
• Resolved: 17/Dec/20 12:00

Description

The Iceberg format spec defines what types to use for different file formats, e.g.:

https://iceberg.apache.org/spec/#parquet

Impala should follow the specification:

• Strings in Iceberg tables should be annotated with UTF8 (when file format is Parquet)
• Revise fixed(L) <-> CHAR(L) mapping
  • CHAR(L) in Impala is annotated with UTF8
  • fixed(L) is for fixed-length binary data
  • We should probably remove this mapping

UPDATE: A recent pull request added a new Iceberg class, HiveSchemaConverter. We should use this class to convert between Iceberg and Hive schemas.

Issue Links

• relates to: IMPALA-10368 Support required/optional property when creating Iceberg table by Impala