[HIVE-15199] INSERT INTO data on S3 is replacing the old rows with the new ones

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.3.0
• Component/s: Hive
• Assignee: Sergio Peña
• Reporter: Sergio Peña
• Created: 14/Nov/16 20:31
• Updated: 21/Jul/17 18:26
• Resolved: 23/Nov/16 18:45

Description

Any INSERT INTO statement run on S3 tables and when the scratch directory is saved on S3 is deleting old rows of the table.

hive> set hive.blobstore.use.blobstore.as.scratchdir=true;

hive> create table t1 (id int, name string) location 's3a://spena-bucket/t1';

hive> insert into table t1 values (1,'name1');

hive> select * from t1;
1       name1

hive> insert into table t1 values (2,'name2');

hive> select * from t1;
2       name2

Issue Links

• breaks: HIVE-15280 Hive.mvFile() misses the "." char when joining the filename + extension
• is related to: HIVE-16402 Upgrade to Hadoop 2.8.0
• is related to: HIVE-16411 Revert HIVE-15199
• relates to: HADOOP-13823 s3a rename: fail if dest file exists
• relates to: HIVE-12988 Improve dynamic partition loading IV
• links to: Review Board