[HUDI-3082] [Phase 1] Unify MOR table access across Spark, Hive

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Duplicate
• Fix Version/s: 0.11.0
• Labels: pull-request-available
• Story Points: 10
• Assignee: Alexey Kudinkin
• Reporter: Alexey Kudinkin

Reviewers

Ethan Guo (this is the old account; please use "yihua"), Shiyan Xu

• Created: 21/Dec/21 04:32
• Updated: 20/Jan/22 21:56
• Resolved: 12/Jan/22 21:43

Description

This is Phase 1 of what outlined in HUDI-3081

 

The goal is 

• Unify Hive’s RecordReaders (`RealtimeCompactedRecordReader`, RealtimeUnmergedRecordReader)
  • These Readers should only differ in the way they handle the payload, everything else should remain constant
• Abstract w/in common component (name TBD)
  • Listing current file-slices at the requested instant (handling the timeline)
  • Creating Record Iterator for the provided file-slice

Issue Links

• duplicates: HUDI-3081 Revisiting Read Path Infra across Query Engines