[IMPALA-10495] Computing correlation coefficient for certain columns can be useful to min/max filters

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Epic Color: ghx-label-7
• Assignee: Unassigned
• Reporter: Qifan Chen
• Created: 09/Feb/21 16:40
• Updated: 09/Feb/21 17:36

Description

Selecting good min/max filters for a query during query compilation can be a difficult task as the benefit of such a filter may not be known. However, when a column C is known to have a strong correlation with a partition or a sort column, a min/max filter on C can be chosen and very useful.