[HIVE-17296] Acid tests with multiple splits

• Type: Test
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0
• Component/s: Transactions
• Target Version/s: 3.0.0
• Assignee: Unassigned
• Reporter: Eugene Koifman
• Created: 10/Aug/17 23:28
• Updated: 28/Jul/21 14:13

Description

data files in an Acid table are ORC files which may have multiple stripes
for such files in base/ or delta/ (and original files with non acid to acid conversion) are split by OrcInputFormat into multiple (stripe sized) chunks.
There is additional logic in in OrcRawRecordMerger (discoverKeyBounds/discoverOriginalKeyBounds) that is not tested by any E2E tests since none of the have enough data to generate multiple stripes in a single file.

testRecordReaderOldBaseAndDelta/testRecordReaderNewBaseAndDelta/testOriginalReaderPair
in TestOrcRawRecordMerger has some logic to test this but it really needs e2e tests.

With ORC-228 it will be possible to write such tests.

Issue Links

• is blocked by: ORC-228 Make MemoryManagerImpl.ROWS_BETWEEN_CHECKS configurable