Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6148

RunMongoAggregation - Add support for allowDiskUse option

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.9.0
    • Fix Version/s: 1.10.0
    • Component/s: Core Framework
    • Labels:
    • Environment:
      All
    • Flags:
      Patch

      Description

      Currently the RunMongoAggregation processor does not allow the use of the 'allowDiskUse' aggregation option. This is required when aggregation pipeline stages exceed the maximum memory use limit (100MB) and is typically exceeded when working with large datasets particularly when executing $group or $sort operations.

       

      Pull Request will submitted in due course to add a mandatory property on the RunMongoAggregation processor (defaulting to false).

       

      https://docs.mongodb.com/manual/reference/command/aggregate/#aggregate-data-using-external-sort

      http://mongodb.github.io/mongo-java-driver/3.10/javadoc/com/mongodb/client/AggregateIterable.html#allowDiskUse-java.lang.Boolean-

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                cdavid15 Craig Davidson
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h