[HUDI-960] HFile Support for HUDI

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.7.0
• Component/s: Common Core
• Labels: pull-request-available
• Assignee: Prashant Wason
• Reporter: Nishith Agarwal
• Created: 26/May/20 19:13
• Updated: 21/Jan/21 05:34
• Resolved: 31/Aug/20 18:43

Description

This task builds upon HUDI-684 which introduces abstraction to storage reader/writer for base/log file formats. HFIie format is required for RFC-15.

The following items will need to be implemented:

1. HFileWriter and HFileReader. 
2. HoodieHFileDataBlock to allow for logging Hfile to log blocks
3.  HoodieLogFormatWriter and HoodieLogFileReader support for HoodieHFileDataBlock
4. HoodieMergedLogRecordScanner support for HoodieHFileDataBlocks

New unit tests:

1. HFileWriter and HFileReader
2. HoodieHFileDataBlock

Issue Links

• Blocked: HUDI-961 Implement HoodieHFileDataBlock to allow for logging Hfile to log blocks
• links to: GitHub Pull Request #1804