[HUDI-5007] Prevent Hudi from reading the entire timeline's when performing a LATEST streaming read

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.12.2
• Labels: pull-request-available
• Assignee: voon
• Reporter: voon
• Created: 11/Oct/22 06:34
• Updated: 16/Feb/23 06:13
• Resolved: 29/Nov/22 12:52

Description

The current implementation of the Hudi-on-Flink stream reader will attempt to read out all files on the active timeline when reading from the latest instant (read.start-commit not specified).

 

This will cause unnecessary file exists check, decreasing read performance on the first read iteration.

Issue Links

• links to: GitHub Pull Request #6920
• links to: GitHub Pull Request #7471