[IMPALA-7854] Slow ALTER TABLE and LOAD DATA statements for tables with large number of partitions

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 2.12.0
• Component/s: Catalog
• Labels: impala performance
• Environment: 14 Nodes ; Table in question has 20 columns, 3 partition columns, and 57,475 partitions
• Epic Color: ghx-label-4
• Assignee: Vihang Karajgaonkar
• Reporter: vietn
• Created: 15/Nov/18 19:10
• Updated: 13/Feb/20 18:54

Description

ALTER TABLE and LOAD DATA statements take minutes (9 minutes for ALTER TABLE and 6 minutes for LOAD DATA) for tables with a large number of partitions.

Our workaround was to use Hive to perform the LOAD DATA and then perform a REFRESH PARTITION using Impala.

• 14 Nodes
• Table in question has 20 columns, 3 partition columns, and 57,475 partitions