[HUDI-6725] Support efficient completion time queries on the timeline

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0-beta1
• Component/s: core
• Labels: pull-request-available
• Epic Link: Non-blocking concurrency control
• Assignee: Danny Chen
• Reporter: Danny Chen

Reviewers

Vinoth Chandar

• Created: 18/Aug/23 07:21
• Updated: 08/Sep/23 05:40
• Resolved: 08/Sep/23 05:40

Description

The basic idea is we do a eager loading of the completion time on archived timeline, for example, the last 3 days, and all the completed instants of the active timeline.

If a query is asking about a completion time earlier than that time range, just do a lazy look up on the archvied timeline.

 

Probably we would write a completion time loader.

Issue Links

• links to: GitHub Pull Request #9565