Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6972

Stateless NiFi should support parallelism

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.10.0
    • Fix Version/s: None
    • Component/s: NiFi Stateless
    • Labels:

      Description

      Today, when running NiFi in stateless mode, it will run all flow files serially - meaning that only after a flow file finish running the whole flow, it will re run the flow for the next flow file.

      In other words, it runs only one flow file at a time. Is there any reason it was implemented that way?

      Instead, I think that there should be a new parameter when running the NiFi, which would be the maximum parallelism. In my opinion, it could be implemented quite easily:

      Under StatelessFlow.java, in the run(final Queue<InMemoryFlowFile> output) method, in line 322, it should create a new thread each iteration, with the maximum being the given parameter.

      This will allow the flow to run multiple flow file in parallel, and should be simple enough without breaking anything else...

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              AdamT Adam Trump
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: