[NIFI-6148] RunMongoAggregation - Add support for allowDiskUse option

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.9.0
• Fix Version/s: 1.10.0
• Component/s: Core Framework
• Labels: MongoDB Processor
• Environment: All
• Flags: Patch
• Assignee: Unassigned
• Reporter: Craig Davidson
• Created: 25/Mar/19 21:31
• Updated: 28/Mar/19 15:10
• Resolved: 28/Mar/19 15:10

Description

Currently the RunMongoAggregation processor does not allow the use of the 'allowDiskUse' aggregation option. This is required when aggregation pipeline stages exceed the maximum memory use limit (100MB) and is typically exceeded when working with large datasets particularly when executing $group or $sort operations.

 

Pull Request will submitted in due course to add a mandatory property on the RunMongoAggregation processor (defaulting to false).

 

https://docs.mongodb.com/manual/reference/command/aggregate/#aggregate-data-using-external-sort

http://mongodb.github.io/mongo-java-driver/3.10/javadoc/com/mongodb/client/AggregateIterable.html#allowDiskUse-java.lang.Boolean-

 

 

Issue Links

• links to: GitHub Pull Request #3389