[SPARK-21975] Histogram support in cost-based optimizer

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.3.0
• Component/s: SQL
• Labels: releasenotes
• Assignee: Zhenhua Wang
• Reporter: Zhenhua Wang
• Created: 11/Sep/17 12:10
• Updated: 01/Apr/18 09:18
• Resolved: 21/Dec/17 13:40

Issue Links

• Is contained by: SPARK-16026 Cost-based Optimizer Framework
• is related to: SPARK-23839 consider bucket join in cost-based JoinReorder rule

Sub-Tasks

• generate equi-height histogram for column: Resolved
• Make percentile_approx support date/timestamp type and change the output type to be the same as input type: Resolved
• Aggregation function for counting distinct values for multiple intervals: Resolved
• Change implementation of ApproxCountDistinctForIntervals to TypedImperativeAggregate: Resolved
• support histogram in filter cardinality estimation: Resolved
• Refactor join estimation to incorporate estimation logic for different kinds of statistics: Resolved
• Use histogram stats in join estimation: Resolved
• support histogram in estimating the cardinality of aggregate (or group-by) operator: Closed
• show histogram in DESC COLUMN command: Resolved
• Should not try to generate histogram for empty/null columns: Resolved