[IMPALA-592] Improve performance of ALTER TABLE ADD PARTITION

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 1.1.1
• Fix Version/s: Impala 1.2
• Assignee: Lenni Kuff
• Reporter: Lenni Kuff
• Created: 17/Sep/13 15:53
• Updated: 16/Oct/13 01:32
• Resolved: 16/Oct/13 01:32

Description

Each time an "ALTER TABLE ADD PARTITION" statement is executed, the table metadata is partially invalidated and reloaded. For tables with few partitions, this is relatively fast (<1 second), but increases linearly over time with the number of partitions. A table with 2000 partitions can take 20s+ to complete execution of a single ALTER TABLE ADD PARTITION statement. We should fix this by skipping the invalidation/reload step and instead directly applying the metadata changes to the HdfsTable object.