[HIVE-25502] Cleaner causes data loss when processing aborted txn with DP

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: Denys Kuzmenko
• Reporter: Denys Kuzmenko
• Created: 06/Sep/21 12:05
• Updated: 17/Nov/22 08:48
• Resolved: 10/Sep/21 09:05

Description

Steps to reproduce:
1) CREATE TABLE cws (b STRING) PARTITIONED BY (a INT) STORED AS ORC;
2) insert into cws partition (a) values (1, '1');
update csw set b='2' where a=1;
3) open HiveStreamingConnection with batch size set to 1, begin txn, write something and abort the txn
4) insert into cws partition (a) values (2, '2');
update csw set b='3' where a=2;
5) manually trigger the compaction and verify that data for part a=1 is gone

Issue Links

• is caused by: HIVE-21052 Make sure transactions get cleaned if they are aborted before addPartitions is called
• links to: GitHub Pull Request #2619