[IMPALA-2664] prevent INCREMENTAL STATS to run when total size would exceed a threshold

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.3.0
• Fix Version/s: Impala 2.5.0, Impala 2.3.2, Impala 2.2.9
• Target Version: Impala 2.5.0, Impala 2.3.2, Impala 2.2.9
• Assignee: Bharath Vissapragada
• Reporter: Silvius Rus
• Created: 13/Nov/15 22:45
• Updated: 23/Jun/16 17:56
• Resolved: 23/Nov/15 06:14

Description

Computing incremental stats can take a large amount of space in the metastore and then on the wire between catalogd and impalad, which can lead to slowness or catalog crashes.

Impala should fail COMPUTE INCREMENTAL STATS when the (estimated) size of incremental stats present in the metastore exceeds a given threshold.