[HIVE-28047] Iceberg: Major QB Compaction on unpartitioned tables with a single commit

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.1.0
• Component/s: Hive, Iceberg integration
• Labels: pull-request-available
• Assignee: Dmitriy Fingerman
• Reporter: Dmitriy Fingerman
• Created: 30/Jan/24 15:08
• Updated: 15/Aug/24 02:06
• Resolved: 13/Aug/24 09:22

Description

Currently Hive Iceberg Major QB Compaction happens in 2 commits in a single transaction. First commit deletes all existing files and second commit inserts compacted data files. If a user queries the table using snapshot id of the first commit after the compaction, there will be no records returned which is invalid. Need to find a way to do it in one commit.

Issue Links

• links to: GitHub Pull Request #5389