[HIVE-27965] Table/partition rename takes a long time at PART_COL_STATS for wide tables

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.0.0
• Assignee: Unassigned
• Reporter: Naresh P R
• Created: 21/Dec/23 17:50
• Updated: 15/Jul/24 05:58

Description

Partition table rename gets clogged at PART_COL_STATS for wide tables.

CREATE TABLE IF NOT EXISTS `PART_COL_STATS` (
 ...
 `DB_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
 `TABLE_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
 `PARTITION_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL, 
  ...)

Since PART_COL_STATS holds db_name & table_name, incase of table rename, every row in PART_COL_STATS associated with the table should be fetched, stored in memory, delete & re-insert with new db/table/partition name.

 

Remove DB_NAME, TABLE_NAME, PARTITION_NAME from PART_COL_STATS, instead use TBL_ID, DB_ID, PART_ID to avoid touching PART_COL_STATS for table/partition renames.

Also TBL_ID, DB_ID, PART_ID can be used for PART_COL_STATS INDEXING.

Issue Links

• is related to: HIVE-28372 No need to update partitions stats when renaming table
• relates to: HIVE-27725 Remove redundant columns in TAB_COL_STATS and PART_COL_STATS