[NIFI-9544] LookUpRecord does not work for arrays if look up value is missing for a key in the array

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.16.0
• Component/s: Extensions
• Environment: apache/nifi:1.15.0 official docker image
• Assignee: Pierre Villard
• Reporter: Peter Molnar
• Created: 06/Jan/22 14:52
• Updated: 13/Jan/22 21:23
• Resolved: 13/Jan/22 21:23

Description

I tried to use the LookUpRecord processor with the example shown on https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.15.0/org.apache.nifi.processors.standard.LookupRecord/additionalDetails.html page under "Record Update Strategy - Replace Existing Values" section.

In the Simple Key Value Lookup Service I defined the following key-value pairs (Note: that FR key is missing on purpose):

• US => United States
• CA => Canada

 

I created a GenerateFlowFile processor to create a file with the following content:

[
    {
        "locales": [
           

{                 "region": "FR",                 "language": "fr"             }

,

{                 "region": "US",                 "language": "en"             }

        ]
    }, {
        "locales": [
           

{                 "region": "CA",                 "language": "fr"             }

, 
           

{                 "region": "JP",                 "language": "ja"             }

        ]
    }
]

 

The LookUpRecord is configured as follows:

JsonTreeReader and JsonRecordSetWriter are just configured with infer and inherit schema.

Once LookUpRecord is executed, "US" values is not looked up and replaced. The reason for this since "FR" cannot be looked up, line 343 in LookUpRecord.java (https://github.com/apache/nifi/blob/7fe8858d22f580ad9dcd96ebafb59dbe5e73f5cb/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LookupRecord.java#L341) returns before attempting to look up the "US" value.

I would expect that in this case "US" is still looked up regardless if "FR" key can be looked up or not.

 

Please find enclosed the NiFi template for reproducing the issue.

 

Issue Links

• links to: GitHub Pull Request #5638