[HIVE-14018] Make IN clause row selectivity estimation customizable

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.0, 2.2.0
• Fix Version/s: 2.1.1, 2.2.0
• Component/s: Statistics
• Labels: TODOC2.1.1 TODOC2.2
• Assignee: Jesús Camacho Rodríguez
• Reporter: Jesús Camacho Rodríguez
• Created: 15/Jun/16 16:18
• Updated: 27/Feb/24 22:23
• Resolved: 17/Jun/16 11:16

Description

After HIVE-13287 went in, we calculate IN clause estimates natively (instead of just dividing incoming number of rows by 2). However, as the distribution of values of the columns is considered uniform, we might end up heavily underestimating/overestimating the resulting number of rows.

This issue is to add a factor that multiplies the IN clause estimation so we can alleviate this problem. The solution is not very elegant, but it is the best we can do until we have histograms to improve our estimate.

Issue Links

• relates to: HIVE-13287 Add logic to estimate stats for IN operator