[NIFI-11891] Add No Tracking Strategy to ListGCS

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0-M1, 1.24.0
• Component/s: Extensions
• Assignee: Pierre Villard
• Reporter: Noura Ali
• Created: 01/Aug/23 07:42
• Updated: 23/Nov/23 15:13
• Resolved: 07/Aug/23 19:08

Description

Currently, when ListGCS runs for the first time, it lists all the available files in the specified bucket directory, and when it runs in the next times, it lists the files created/modified since last run, using the state stored in the processor.

What is required is that every time the processor runs(runs after first one), to list all existing files in GCS specified directory, not the files created/modified since last run.

As I understand this is called "No Tracking" listing strategy that is available in most List* processors, but it doesn't exist in ListGCSBucket.
 

Without this capability, I had to build a whole extra pipeline to stop the processor–> clear the state -->start processor, before the ListGCS runs, which is prone to issues with extra work and maintenance effort. 
 

Appreciate your help and support.

Issue Links

• links to: GitHub Pull Request #7570