[HUDI-4927] GCS Ingestion supports only new file uploads, no deletion and repeated uploads

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: deltastreamer
• Epic Link: Implement DeltaStreamer Source for cloud object stores
• Assignee: Unassigned
• Reporter: Pramod Biligiri
• Created: 27/Sep/22 10:21
• Updated: 10/Mar/23 01:50

Description

The GCS Ingestion (https://issues.apache.org/jira/browse/HUDI-4850) supports only events related to new files which are being uploaded for the first time. Specifically, it does not detect files being deleted, or the same file being uploaded repeatedly.

GCS even has a notion of Object Versioning, which is also not supported.

Issue Links

• is a child of: HUDI-4850 Implement DeltaStreamer Source for Google Cloud Storage