[SPARK-21079] ANALYZE TABLE fails to calculate totalSize for a partitioned table

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.1
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Maria
• Reporter: Maria
• Created: 13/Jun/17 14:34
• Updated: 10/Jul/17 23:29
• Resolved: 25/Jun/17 05:53

Description

ANALYZE TABLE table COMPUTE STATISTICS invoked for a partition table produces totalSize = 0.

AnalyzeTableCommand fetches table-level storage URI and calculated total size of files in the corresponding directory recursively. However, for partitioned tables, each partition has its own storage URI which may not be a subdirectory of the table-level storage URI.

Issue Links

• links to: [Github] Pull Request #18309 (mbasmanova)