[NIFI-4084] MergeContent with Avro merge format - transfer relationship not specified

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.4.0
• Component/s: Extensions
• Assignee: Unassigned
• Reporter: Pierre Villard
• Created: 19/Jun/17 08:15
• Updated: 06/Jul/18 13:19

Description

Using MergeContent processor with Bin-Packing strategy and Avro merge format. Sometimes I see the following error:

2017-06-19 09:38:24,531 WARN [Timer-Driven Process Thread-14] o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=8dae064d-c934-4292-8ecb-79d77da326d8,claim=,offset=0,name=1822112642788109,size=0] transfer relationship not specified
        at org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:248)
        at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:318)
        at org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:239)
        at org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:191)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
2017-06-19 09:38:25,568 ERROR [Timer-Driven Process Thread-6] o.a.n.processors.standard.MergeContent MergeContent[id=1b013987-39bd-1587-8f89-244f075fff50] Failed to process bundle of 624 files due to org.apache.nifi.processor.exception.ProcessException: IOException thrown from MergeContent[id=1b013987-39bd-1587-8f89-244f075fff50]: java.io.IOException: Not a data file.: {}
org.apache.nifi.processor.exception.ProcessException: IOException thrown from MergeContent[id=1b013987-39bd-1587-8f89-244f075fff50]: java.io.IOException: Not a data file.
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2159)
        at org.apache.nifi.processors.standard.MergeContent$AvroMerge$1.process(MergeContent.java:895)
        at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
        at org.apache.nifi.processors.standard.MergeContent$AvroMerge.merge(MergeContent.java:890)
        at org.apache.nifi.processors.standard.MergeContent.processBin(MergeContent.java:462)
        at org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:231)
        at org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:191)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Not a data file.
        at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:105)
        at org.apache.avro.file.DataFileStream.<init>(DataFileStream.java:84)
        at org.apache.nifi.processors.standard.MergeContent$AvroMerge$1$1.process(MergeContent.java:899)
        at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2136)
        ... 17 common frames omitted
2017-06-19 09:38:25,569 ERROR [Timer-Driven Process Thread-6] o.a.n.processors.standard.MergeContent MergeContent[id=1b013987-39bd-1587-8f89-244f075fff50] MergeContent[id=1b013987-39bd-1587-8f89-244f075fff50] failed to process session due to org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=5cac1983-db59-4acb-b6c6-7b3ac0f579ef,claim=,offset=0,name=1822113685824782,size=0] transfer relationship not specified: {}

Not sure why it happens yet (still looking into it) but could be because of flow files with no content?