Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Won't Do
-
3.2.0
-
None
-
None
Description
AlterTableAddPartitionCommand.run does almost the same as `CommandUtils.updateTableStats` while updating table stats. The code:
if (table.stats.nonEmpty) { if (sparkSession.sessionState.conf.autoSizeUpdateEnabled) { val addedSize = CommandUtils.calculateMultipleLocationSizes(sparkSession, table.identifier, parts.map(_.storage.locationUri)).sum if (addedSize > 0) { val newStats = CatalogStatistics(sizeInBytes = table.stats.get.sizeInBytes + addedSize) catalog.alterTableStats(table.identifier, Some(newStats)) } } else { catalog.alterTableStats(table.identifier, None) } }
can be replaced by:
CommandUtils.updateTableStats()
Attachments
Attachments
Issue Links
- is fixed by
-
SPARK-34084 ALTER TABLE .. ADD PARTITION does not update table stats
- Resolved
- links to