[HIVE-27725] Remove redundant columns in TAB_COL_STATS and PART_COL_STATS

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0, 4.0.0-beta-1
• Fix Version/s: 4.1.0
• Component/s: Hive
• Labels: pull-request-available
• Assignee: Wechar
• Reporter: Wechar
• Created: 24/Sep/23 13:49
• Updated: 15/Jul/24 06:09
• Resolved: 09/Apr/24 14:52

Description

TAB_COL_STATS table includes CAT_NAME, DB_NAME and TABLE_NAME, which can be fetched by join TBLS and DBS tables on TBL_ID and DB_ID columns.

PART_COL_STATS table includes CAT_NAME, DB_NAME, TABLE_NAME and PARTITION_NAME, which can be fetched by join PARTITIONS, TBLS and DBS tables on PART_ID, TBL_ID and DB_ID.

In addition, current HMS get table statistics without join other table, while delete table statistics with join TBLS. This inconsistency will result exception if in a corner case where some table column statistics were left when drop table, then the user recreate the table with same name and database name but will get another TBL_ID, in this case user will get the old table column statistics incorrectly. And if user try delete stats fetched by get api, the NoSuchObjectException will be thrown.

Issue Links

• is depended upon by: HIVE-28372 No need to update partitions stats when renaming table
• is related to: HIVE-27965 Table/partition rename takes a long time at PART_COL_STATS for wide tables
• relates to: HIVE-6190 redundant columns in metastore schema for stats
• links to: GitHub Pull Request #4744