[IMPALA-2423] Concurrent operations on the same partitioned table may cause stale partition metadata

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 2.2.4, Impala 2.3.0
• Component/s: Catalog
• Labels: catalog-server
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Dimitris Tsirogiannis
• Created: 25/Sep/15 17:32
• Updated: 06/Oct/18 21:47

Description

The following operations, when being executed concurrently on the same partitioned table, may cause stale partition metadata.

Query: insert OVERWRITE part_table partition (id='1') SELECT * from table where id = 1;
Inserted 4205565 row(s) in 8.50s
Query: alter TABLE part_table PARTITION (id='1') SET tblproperties('numRows'='3604700', 'STATS_GENERATED_VIA_STATS_TASK' ='true')
ERROR: AnalysisException: Partition spec does not exist: (id='1').

The operations are executed on the same impalad node and every INSERT, ALTER TABLE sequence is targeting a different partition. The problem appears with 10-20 concurrent operations.

Issue Links

• is related to: IMPALA-1628 Improve synchronization protocol in the catalog