[IMPALA-12257] createInsertEvents failed by NullPointerException: Invalid partition name

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: Impala 4.3.0
• Component/s: Catalog
• Epic Color: ghx-label-7
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 30/Jun/23 06:59
• Updated: 16/Aug/23 06:16
• Resolved: 16/Aug/23 06:16

Description

INSERT on partitioned table could fail in createInsertEvents() if some updated partitions are missing in catalogd but actually exists in HMS. The failure is a NullPointerException, e.g.

I0630 14:44:29.779798 30287 jni-util.cc:288] java.lang.NullPointerException: Invalid partition name: p=0
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:907)
        at org.apache.impala.catalog.HdfsTable.getPartitionsForNames(HdfsTable.java:1758)
        at org.apache.impala.service.CatalogOpExecutor.createInsertEvents(CatalogOpExecutor.java:6935)
        at org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:6830)
        at org.apache.impala.service.JniCatalog.lambda$updateCatalog$16(JniCatalog.java:471)
        at org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
        at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
        at org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
        at org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
        at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:230)
        at org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:470) 

We've seen this in an issue caused by IMPALA-12256 in which the stale DROP_PARTITION event incorrectly drops the partition, causing inconsistency between catalogd and HMS on the partition list.

To steadily reproduce the issue, we can disable HMS event-processing and manually make the partition list differs between catalogd and HMS.

Start Impala with HMS event-processing disabled and create a partitioned table. Run a query on it to make it loaded in catalogd:

bin/start-impala-cluster.py --catalogd_args=--hms_event_polling_interval_s=0

impala> create table my_part (id int) partitioned by (p int) stored as textfile;
impala> show partitions my_part;

Add one partition in Hive. Catalogd is not aware of it:

hive> alter table my_part add partition(p=0);

Then run INSERT on the same partition in Impala:

impala> insert into my_part partition(p=0) values (0);
ERROR: NullPointerException: Invalid partition name: p=0

Issue Links

• links to: Code Review