[HUDI-4034] Improve log merging performance for Metadata Table

• Type: Task
• Status: In Progress
• Priority: Blocker
• Resolution: Unresolved
• Component/s: metadata, performance
• Story Points: 4
• Epic Link: Performance Improvements
• Assignee: Ethan Guo (this is the old account; please use "yihua")
• Reporter: Ethan Guo (this is the old account; please use "yihua")
• Created: 03/May/22 22:47
• Updated: 10/Mar/23 01:02

Description

For Metadata Table as a MOR table, there is a separate log record reader "HoodieMetadataMergedLogRecordReader".  We want to see if we can further improve the merging latency on different file systems including S3.