[HIVE-25293] Alter partitioned table with "cascade" option create too many columns records.

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.3, 3.1.2
• Component/s: Metastore
• Labels: pull-request-available
• Assignee: yongtaoliao
• Reporter: yongtaoliao
• Created: 27/Jun/21 13:33
• Updated: 06/Sep/21 00:11

Description

When alter partitioned table with "cascade" option, all partitions supports to be updated. Currently, a CD_ID will be created for each partition, associated with a set of Columns, which will cause a large amount of redundant data in the metadata database.

The following DDL statements can reproduce this scenario：

 

create table test_table (f1 int) partitioned by (p string);
alter table test_table add partition(p='a');
alter table test_table add partition(p='b');
alter table test_table add partition(p='c');
alter table test_table add columns (f2 int) cascade;

All partitions use the table's `CD_ID` before adding columns, while each partition use their own `CD_ID` after adding columns.

 

My proposal is all partitions should use the same `CD_ID` when table was altered with "cascade" option.

Issue Links

• links to: GitHub Pull Request #2434
• links to: GitHub Pull Request #2436