[IMPALA-11210] Impala can only handle lowercase schema elements of Iceberg table

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.1.0
• Component/s: Catalog
• Labels: impala-iceberg
• Epic Color: ghx-label-3
• Assignee: Zoltán Borók-Nagy
• Reporter: Zoltán Borók-Nagy
• Created: 30/Mar/22 13:25
• Updated: 31/Mar/22 21:45
• Resolved: 31/Mar/22 21:45

Description

When Impala/Hive creates a table they lowercase the schema elements.

Impala even has precondition checks that the schema elements are lower case:
https://github.com/apache/impala/blob/b1c1be12f3cceed48e93eddae8b9512737e3e0d2/fe/src/main/java/org/apache/impala/catalog/Column.java#L56

But when an Iceberg table is created via Spark it doesn't lowercase the column names, so the metadata.json file looks like this:

...
  "schema" : {
    "type" : "struct",
    "schema-id" : 0,
    "fields" : [ {
      "id" : 1,
      "name" : "Region",
      "required" : false,
      "type" : "string"
    }, {
...

Hence when Impala tries to load such tables it runs to the above precondition check which raises an IllegalStateException.