[NIFI-8950] Allow Stateless NiFi to use a file system backed Content Repository

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: NiFi Stateless
• Assignee: Mark Payne
• Reporter: Mark Payne
• Created: 27/Jul/21 21:16
• Updated: 11/Aug/21 17:09
• Resolved: 11/Aug/21 17:09

Description

Stateless NiFi is capable of storing all FlowFile content on Java Heap. This is often what we want to do, as often times this is used for small 'message' types of flows. However, there are times when we do not want to do this. For example, consider a scenario in which we want to download a file from S3 and push it to some other S3 bucket. If that file is several GB, we may not have the resources to store the entire contents in Java heap.

Instead, we may want to store the data in 'scratch space'. This change would NOT result in data being persistent across restarts of the application - or even across successive triggerings of the dataflow. I.e., this is not intended to provide stateful, persistent storage of the data, but rather to allow for larger FlowFiles than what can fit in memory to still be operated on by Stateless NiFi

Issue Links

• links to: GitHub Pull Request #5254