[IMPALA-5300] Implement TABLESAMPLE

• Type: New Feature
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 2.12.0
• Component/s: Frontend
• Labels: compute-stats observability
• Target Version: Impala 2.12.0
• Epic Color: ghx-label-7
• Assignee: Alexander Behm
• Reporter: Alexander Behm
• Created: 10/May/17 17:26
• Updated: 04/Feb/18 01:42
• Resolved: 04/Feb/18 01:42

Description

Implement the TABLESAMPLE clause that can be used against base table references in queries as well as the COMPUTE STATS statement.

Examples:

SELECT * FROM T TABLESAMPLE SYSTEM(10)
COMPUTE STATS T TABLESAMPLE SYSTEM(20)

Syntax inspired by SQL Server:
https://technet.microsoft.com/en-us/library/ms189108(v=sql.105).aspx

<tableref> TABLESAMPLE SYSTEM(<number>) [REPEATABLE(<number>)]

Implementation details

• The given percentage refers to the percent of bytes in the table.
• The sampling will be coarse-grained (file level).
• Impala will randomly select files until the desired percentage of bytes has been reached

Accepted limitations

• Computing stats on a coarse-grained sample necessarily means a loss of precision with no guarantee on statistical significance
• There is no guarantee that a sample covers all partitions
• NDVs may be very inaccurate for sorted files
• NDVs may be very inaccurate for an unfortunate selection of files

Sub-Tasks

• Implement TABLESAMPLE for HDFS tables: Resolved
• Implement TABLESAMPLE for COMPUTE STATS: Resolved
• Add minimum sample size for COMPUTE STATS TABLESAMPLE: Resolved
• More flexible configuration of stats extrapolation: Resolved
• Doc: TABLESAMPLE for COMPUTE STATS: Closed
• Impala Doc: Doc the minimum sample size for COMPUTE STATS TABLESAMPLE: Closed