Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-8950

Allow Stateless NiFi to use a file system backed Content Repository

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.15.0
    • Component/s: NiFi Stateless
    • Labels:
      None

      Description

      Stateless NiFi is capable of storing all FlowFile content on Java Heap. This is often what we want to do, as often times this is used for small 'message' types of flows. However, there are times when we do not want to do this. For example, consider a scenario in which we want to download a file from S3 and push it to some other S3 bucket. If that file is several GB, we may not have the resources to store the entire contents in Java heap.

      Instead, we may want to store the data in 'scratch space'. This change would NOT result in data being persistent across restarts of the application - or even across successive triggerings of the dataflow. I.e., this is not intended to provide stateful, persistent storage of the data, but rather to allow for larger FlowFiles than what can fit in memory to still be operated on by Stateless NiFi

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                markap14 Mark Payne
                Reporter:
                markap14 Mark Payne
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m