[HIVE-20738] Enable Delete Event filtering in VectorizedOrcAcidRowBatchReader

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Transactions
• Target Version/s: 4.0.0
• Assignee: Eugene Koifman
• Reporter: Eugene Koifman
• Created: 12/Oct/18 19:57
• Updated: 17/Nov/22 08:54
• Resolved: 20/Nov/18 17:32

Description

Currently DeleteEventRegistry loads all delete events which can take time and use a lot of memory. Should minimize the number of deletes loaded based on the insert events included in the Split.
This is an umbrella jira for several tasks that make up the work. See individual tasks for details.

Issue Links

• is depended upon by: HIVE-20699 Query based compactor for full CRUD Acid tables
• supercedes: HIVE-19955 ACID: Pre-filter the delete event registry using insert delta ranges

Sub-Tasks

• ColumnizedDeleteEventRegistry.DeleteReaderValue optimization: Closed
• VectorizedOrcAcidRowBatchReader.checkBucketId() should run for unbucketed tables: Resolved
• VectorizedOrcAcidRowBatchReader doesn't filter delete events: Closed
• VectorizedOrcAcidRowBatchReader doesn't filter delete events for original files: Closed
• Potential ArrayIndexOutOfBoundsException in VectorizedOrcAcidRowBatchReader.findMinMaxKeys: Closed
• Additional unit tests for VectorizedOrcAcidRowBatchReader min max key evaluation: Closed
• Do delete event filtering even if hive.acid.index is not there: Closed