[IMPALA-4967] Metadata distribution should be incremental

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: Impala 2.9.0
• Component/s: Catalog
• Assignee: Unassigned
• Reporter: Peter Ebert
• Created: 22/Feb/17 20:41
• Updated: 11/Aug/17 21:59
• Resolved: 11/Aug/17 21:59

Description

When running REFRESH table partition (x=1) the metadata distribution contains all partitions, not just the single partition that was refreshed. This creates additional load and delays refreshes.

To solve metadata updates might have to be serialized (assign unique #, and apply in order), but this behavior would help the timeliness of data and reduce load on the cluster.

Issue Links

• is duplicated by: IMPALA-3234 Catalog should send incremental metadata changes to Impalads