[HIVE-7589] Some fixes and improvements to statistics annotation rules

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.14.0
• Fix Version/s: 0.14.0
• Component/s: Query Processor, Statistics
• Assignee: Prasanth Jayachandran
• Reporter: Prasanth Jayachandran
• Created: 01/Aug/14 02:47
• Updated: 13/Nov/14 19:41
• Resolved: 06/Aug/14 17:58

Description

FIXES:
1) JOIN rule does not properly propagate the column statistics from its parent
2) Multi-way join rule computes the denominator for #rows estimation wrongly
3) GROUPBY rule does not account for the data size of aggregate column
4) Prefix removal from column names isn't working
5) GROUPBY rule looks at missing column statistics for aggregate column from its parent and assumes PARTIAL column stats state

IMPROVEMENTS:
1) Replace "EXPLAIN EXTENDED" with "EXPLAIN" in test cases to make the golden files easy to comprehend and to reduce verbosity
2) Introduce rule for ReduceSink operator which only does renaming of column statistics as per output row schema
3) Add more rows to the test datasets to avoid 0 row scenario in join test cases
4) JOIN rule improvement to avoid long overflow

Issue Links

• is related to: HIVE-7156 Group-By operator stat-annotation only uses distinct approx to generate rollups
• links to: Review Board