[IMPALA-6822] Provide a query option to not shuffle on distinct exprs

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.10.0, Impala 2.11.0, Impala 3.0, Impala 2.12.0, Impala 2.13.0
• Fix Version/s: Impala 3.0, Impala 2.12.0
• Component/s: Frontend
• Labels: performance planner regression
• Epic Color: ghx-label-3
• Assignee: Tianyi Wang
• Reporter: Tianyi Wang
• Created: 06/Apr/18 22:21
• Updated: 18/Apr/18 20:06
• Resolved: 17/Apr/18 21:43

Description

After IMPALA-4794, in a distinct aggregation, data will be shuffled on grouping exprs and distinct expr. It works well if the NDV of grouping exprs is low, but is an regression otherwise. We should provide a query operation to disable IMPALA-4794 and probably look to do smarter planning in the future.

Issue Links

• is caused by: IMPALA-4794 Impala's count(distinct ...) plans are not robust to data skew

Sub-Tasks

• Impala 2.12 & 3.0 Docs: Provide a query option to not shuffle on distinct exprs: Closed