[NIFI-7510] MergeRecords not working after RouteOnAttribute

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.11.4
• Environment: MS Windows 10
• Assignee: Unassigned
• Reporter: Shane Downey
• Created: 05/Jun/20 06:54
• Updated: 06/Jun/20 23:40

Description

Here's the scenario:

I need to process an arbitrary length spreadsheet or CSV file.

For each row in the file, I need to extract a value from a column called "Key" and pass it into a SQL Query. If the query returns 0 rows I need to write the source "Key" into an exception file. 

For the successful queries, I need to write the full output to a spreadsheet file.

 

Setup:

GetFile->ConvertExceltoCSVProcessor>SplitRecord(CSVReader/JSONRecordSetWriter)>

ExtractText("Key"/Value)->UpdateAttribute("arg.1"=value)>

ExecuteSQLRecord()->RouteOnAttribute(rowcount=0)>LogAttribute

RouteOnAttribute(rowcount!=0)->MergeRecords(Defragment)>PutFile

 

In the above scenario, the FlowFile blocks in the queue leading into MergeRecords (never actually makes it into MergeRecords).

 

If I change the merge strategy to Bin then data does flow through to the output file in "chunks" of rows.

If I remove RouteOnAttribute from the flow, the solution works as expected and my output file is created as expected.

 

Thus - there appears to be an issue using RouteOnAttribute within a SplitRecord/MergeRecord block and using the defragment merge strategy.