Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-4122

GetMongo should be able to group results into a set of flowfiles

    XMLWordPrintableJSON

    Details

      Description

      GetMongo should be able to take a user-defined limit and group results by that size into flowfiles rather than having only the ability to do a 1:1 relationship between result and flowfile.

      For example, if the user specifies 100, 100 results should be grouped together and turned into a JSON array that can be broken up later as needed.

      This need arose when doing a bulk data ingestion from Mongo. We had shy of 400k documents, and the 1:1 generation of flowfiles blew right through our limits on the content repository. Adding this feature would make it feasible to control that sort of behavior more thoroughly for events like bulk ingestion.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mike.thomsen Mike Thomsen
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: