[IMPALA-6228] More flexible configuration of stats extrapolation

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.10.0, Impala 2.11.0
• Fix Version/s: Impala 2.12.0
• Component/s: Frontend
• Target Version: Impala 2.12.0
• Epic Color: ghx-label-2
• Assignee: Alexander Behm
• Reporter: Alexander Behm
• Created: 21/Nov/17 05:26
• Updated: 04/Feb/18 01:42
• Resolved: 04/Feb/18 01:42

Description

For stats extrapolation (IMPALA-2373) and COMPUTE STATS TABLESMAPLE (IMPALA-5310) we currently require an impalad startup option -enable_stats_extrapolation to be set.

It would be nice if changing that configuration would not require a service restart.
For example, we could consider a query option instead of adding a table property to tables where extrapolation should be enabled.

The reason for the current behavior is as follows:
It is technically not required to be a startup option, but it reduces the number of ways users can shoot themselves in the foot. For example, first running COMPUTE STATS TABLESAMPLE on a table T and then running a query against table T without stats extrapolation does not make sense and will not work well. This subtle behavior might not be clear to users. Yes, that can be addressed with warnings etc., but preventing non-sensical combinations seems better until we have strong evidence against that conservative approach.

Issue Links

• is related to: IMPALA-2373 Extrapolate the number of rows in a scan based on the rows/byte ratio