[DRILL-7365] Failed to read column added to existing Hive partition

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Storage - Hive
• Assignee: Unassigned
• Reporter: Igor Guzenko
• Created: 04/Sep/19 08:37
• Updated: 04/Sep/19 08:38

Description

Prerequisities:

Enable ACID in Hive https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions.

Steps to reproduce:

1) create table hive_bucketed2 (emp_id int, first_name string) PARTITIONED BY (`col_year_month` string) clustered by (emp_id) into 4 buckets stored as orc tblproperties ('transactional'='true');
2) insert into hive_bucketed2 PARTITION (col_year_month = '2019-09') values (1, 'A'),(2, 'B');
3) alter table hive_bucketed2 add columns (age INT);
4) insert into hive_bucketed2 PARTITION (col_year_month = '2019-09') values (11, '1A', 10),(12, '1B', 22);
5) select * from hive.hive_bucketed2;

Workaround (may be a little bit risky :

1. Connect to Hive metastore database.

https://analyticsanvil.files.wordpress.com/2016/08/hive_metastore_database_diagram.png

2. Find SDS linked to desired PARTITIONS . Actually you need CD_ID's for such SDS.

3. Insert your column into COLUMNS_V2 with CD_ID found at previous step.