[SPARK-18911] Decouple Statistics and CatalogTable

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Zhenhua Wang
• Reporter: Zhenhua Wang
• Created: 17/Dec/16 05:26
• Updated: 24/Dec/16 07:36
• Resolved: 24/Dec/16 07:36

Description

Statistics in LogicalPlan should use attributes to refer to columns rather than column names, because two columns from two relations can have the same column name. But CatalogTable doesn't have the concepts of attribute or broadcast hint in Statistics. Therefore, putting Statistics in CatalogTable is confusing. We need to define a different statistic structure in CatalogTable, which is only responsible for interacting with metastore, and is converted to statistics in LogicalPlan when it is used.

Issue Links

• links to: [Github] Pull Request #16323 (wzhfy)