[NIFI-10893] PutMongo doesn't seem to support arrayFilters

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.16.3
• Component/s: Extensions
• Labels: MongoDB
• Environment: Official Docker
• Assignee: Unassigned
• Reporter: Benji Benning
• Created: 29/Nov/22 08:10
• Updated: 29/Nov/22 08:10

Description

Using PutMongo set to Update with Operators enabled, fails when using arrayFilters.

FYI arrayFilters are used to update nested arrays.

The Update query is a simple match on a key. ( { "matchingkey": "xxxx" } )

The document i'm using for the update is something like this:

{
    "$set": {
        "first.$[levela].second.$[levelb].thiskey": "update_this"
    }
},
{
    "arrayFilters": [ 
        { 
            "levela.mainID": "12345"
        },
        {  
            "levelb.subID": "abcde"
        } ]
}

The error i'm getting is:

2022-11-27 20:31:17,359 ERROR [Timer-Driven Process Thread-226] o.a.nifi.processors.mongodb.PutMongo [PutMongo[id=017b1128-0280-1fa9-8cae-35df4120012f], StandardFlowFileRecord[uuid=97ecb515-6619-4fb2-a39a-625aceedb755,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1669580444646-191, container=default, section=191], offset=749291, length=361],offset=0,name=4486523b-5c3d-49a4-b4a6-a209279ed351,size=361], com.mongodb.MongoWriteException: Write operation error on server mongodbdev1:27017. Write error: WriteError{code=2, message='No array filter found for identifier 'levela' in path 'first.$[levela].second.$[levelb].thiskey'', details={}}.] Failed to insert {} into MongoDB due to {}
com.mongodb.MongoWriteException: Write operation error on server mongodbdev1:27017. Write error: WriteError{code=2, message='No array filter found for identifier 'levela' in path 'first.$[levela].second.$[levelb].thiskey'', details={}}.
    at com.mongodb.client.internal.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:1018)
    at com.mongodb.client.internal.MongoCollectionImpl.executeUpdate(MongoCollectionImpl.java:994)
    at com.mongodb.client.internal.MongoCollectionImpl.updateOne(MongoCollectionImpl.java:579)
    at org.apache.nifi.processors.mongodb.PutMongo.onTrigger(PutMongo.java:233)
    at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
    at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283)
    at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
    at org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63)
    at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)

I've also tried to simplify the Update with moving to 1 object in arrayFilters, same issue. Looks like PutMongo entirely ignores the second part of the update document.

Performing this update using other tools or python (my workaround) works fine.