[SPARK-33825] Is Spark SQL able to auto update partition stats like hive by setting hive.stats.autogather=true

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 3.0.1
• Component/s: Spark Core
• Labels: partitionStat
• External issue URL: https://issues.apache.org/jira/browse/SPARK-28872
• External issue ID: 28872
• Assignee: Unassigned
• Reporter: yang
• Created: 17/Dec/20 09:31
• Updated: 12/Dec/22 18:11
• Resolved: 18/Dec/20 05:24

Description

`spark.sql.statistics.size.autoUpdate.enabled` is only work for table stats update.But for partition stats,I can only update it with `ANALYZE TABLE tablename PARTITION(part) COMPUTE STATISTICS`.So is Spark SQL able to auto update partition stats like hive by setting hive.stats.autogather=true?

Issue Links

• relates to: SPARK-38573 Support Auto Partition Statistics Collection