[HIVE-5921] Better heuristics for worst case statistics estimates for join, limit and filter operator

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.13.0
• Fix Version/s: 0.13.0
• Component/s: Query Processor, Statistics
• Assignee: Prasanth Jayachandran
• Reporter: Prasanth Jayachandran
• Created: 03/Dec/13 01:55
• Updated: 13/Dec/16 21:14
• Resolved: 06/Dec/13 00:02

Description

This is a subtask of HIVE-5369. In worst case (i.e; absence of column statistics) HIVE-5849 improved the basic statistics with heuristics. But the heuristics failed to provide better estimates in few cases. For example: FILTER operator heuristics did not take into account the number of predicates and if the predicate contains partition column. Also, JOIN estimates were too aggressive and was not user configurable.

Issue Links

• is related to: HIVE-6300 Add documentation for stats configs to hive-default.xml.template
• links to: Review Board