[IMPALA-339] Improve table refresh time

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 1.0
• Fix Version/s: Impala 1.1
• Assignee: Alan Choi
• Reporter: Alan Choi
• Created: 03/May/13 22:11
• Updated: 21/Jun/13 02:57
• Resolved: 20/Jun/13 20:50

Description

When new files are added to a partition, Impala needs to refresh the metadata of the entire table to pick up the newly added files. For tables with large number of partitions, the refresh process is very time consuming.

We might be able speed up the refresh process: skip the dir if the modification time didn't change.

Sub-Tasks

• Update last_DdlTime after performing dynamic partition insert (if partitions are created): Resolved