Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.9.2
-
None
Description
Proposing New S3 processor that is a "variation" on the existing ListS3 Processor. For lack of better term, I will call this "Watch Multiple S3 Buckets Processor" (naming here is open to feedback). This new Processor takes an input attribute "Bucket" from the FlowFile, which could use NiFi ExpressionLanguage to compute that value. Hence, the Processor could "watch" different buckets in an S3 Object Store.
This processor must handle the State Management logic for multiple S3 buckets. The simplest solution for this problem add to modify the State Management logic to prepend the bucket name to this key/objectName that is being used to track state.
Finally, the Dev should document the maximum number of buckets it will monitor at once, maximum amount of state/storage it can have to track the various listings in various buckets.
Attachments
Issue Links
- links to
