[HIVE-19891] inserting into external tables with custom partition directories may cause data loss

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Assignee: Sergey Shelukhin
• Reporter: Sergey Shelukhin
• Created: 14/Jun/18 01:36
• Updated: 17/Nov/22 08:54
• Resolved: 23/Jul/18 22:57

Description

tbl1 is just used as a prop to create data, could be an existing directory for an external table.
Due to weird behavior of LoadTableDesc (some ancient code for overriding old partition path), custom partition path is overwritten after the query and the data in it ceases being a part of the table (can be seen in desc formatted output with masking commented out in QTestUtil)
This affects branch-1 too, so it's pretty old.

drop table tbl1;
CREATE TABLE tbl1 (index int, value int ) PARTITIONED BY ( created_date string );
insert into tbl1 partition(created_date='2018-02-01') VALUES (2, 2);

CREATE external TABLE tbl2 (index int, value int ) PARTITIONED BY ( created_date string );
ALTER TABLE tbl2 ADD PARTITION(created_date='2018-02-01');
ALTER TABLE tbl2 PARTITION(created_date='2018-02-01') SET LOCATION 'file:/Users/sergey/git/hivegit/itests/qtest/target/warehouse/tbl1/created_date=2018-02-01';
select * from tbl2;
describe formatted tbl2 partition(created_date='2018-02-01');
insert into tbl2 partition(created_date='2018-02-01') VALUES (1, 1);
select * from tbl2;
describe formatted tbl2 partition(created_date='2018-02-01');

Issue Links

• relates to: HIVE-20603 "Wrong FS" error when inserting to partition after changing table location filesystem