[SPARK-34120] Improve the statistics estimation

• Type: Umbrella
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: SQL
• Assignee: Yuming Wang
• Reporter: Yuming Wang
• Created: 15/Jan/21 03:30
• Updated: 18/Jun/21 13:52
• Resolved: 18/Jun/21 13:50

Description

This umbrella tickets aim to track improvements in statistics estimates.

Sub-Tasks

• Improve the statistics estimation of the Tail: Resolved
• Some operator missing rowCount when enable CBO: Resolved
• Union operator missing rowCount when enable CBO: Resolved
• Add rowCount for Range operator: Resolved
• Keep necessary stats after partition pruning: Resolved
• Intersect operator missing rowCount when CBO enabled: Resolved
• Improve Distinct statistics estimation: Resolved
• Improve Repartition statistics estimation: Resolved