[IMPALA-10398] Altering an Iceberg table might throw NullPointerException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Epic Color: ghx-label-13
• Assignee: Zoltán Borók-Nagy
• Reporter: Zoltán Borók-Nagy
• Created: 16/Dec/20 13:49
• Updated: 17/Dec/20 12:00
• Resolved: 17/Dec/20 12:00

Description

IcebergSchemaConverter has a static thread local member which can be uninitialized in the current thread when nextId() is invoked.
 
In that case the thread local integer's get() method returns a null and we get a NullPointerException when we want to convert it to a builtin int.

I1215 17:25:28.418272 22202 jni-util.cc:288] java.lang.NullPointerException
 at org.apache.impala.util.IcebergSchemaConverter.nextId(IcebergSchemaConverter.java:179)
 at org.apache.impala.util.IcebergSchemaConverter.fromImpalaType(IcebergSchemaConverter.java:162)
 at org.apache.impala.util.IcebergSchemaConverter.fromImpalaColumnType(IcebergSchemaConverter.java:121)
 at org.apache.impala.service.IcebergCatalogOpExecutor.addColumn(IcebergCatalogOpExecutor.java:103)
 at org.apache.impala.service.CatalogOpExecutor.alterIcebergTable(CatalogOpExecutor.java:972)
 at org.apache.impala.service.CatalogOpExecutor.alterTable(CatalogOpExecutor.java:701)
 at org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:370)
 at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:171)

 
Because of this issue test_alter_iceberg_tables is flaky, especially on systems with a high number of CPUs.
 
The error can be reproduced by restarting the Impala cluster and adding complex types to an Iceberg table, e.g.:

ALTER TABLE ice_table
ADD COLUMNS(map_test MAP <STRING, array <STRING>>, struct_test STRUCT <f1: BIGINT, f2: BIGINT>);