[HUDI-6738] Apply object filter before checkpoint batching in GcsEventsHoodieIncrSource

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Lokesh Lingarajan
• Created: 22/Aug/23 22:50
• Updated: 11/Sep/23 17:28
• Resolved: 11/Sep/23 17:28

Description

Recent refactoring to support batching within commit for GCS incr job moved the filtering of  objects after the checkpoint batching. The issue with this on bootstrap scenarios where we are looking for only latest commits, we will have to go through the entire set of commits based on sourcelimit instead of directly skipping to the latest commit. 

Fix is to apply filtering before we start checkpoint batching. This change list will bring GCS job similar to S3 job. 

Issue Links

• links to: GitHub Pull Request #9538