Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-8760

VolatileContentRepository fails to retrieve content from claims with several processors

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      For several processors such as MergeRecord, QueryRecord, SplitJson, the use of VolatileContentRepository implementation infers errors while retrieving Flowfiles from claims. The following logs are generated using NiFi 1.13.1 from Docker and the flow.xml.gz and nifi.properties file attached.

      MergeRecord (with JsonTreeReader, JsonRecordSetWriter with default configuration):

      2021-07-06 10:15:09,170 ERROR [Timer-Driven Process Thread-1] o.a.nifi.processors.standard.MergeRecord MergeRecord[id=7b425cff-017a-1000-6a20-58c4e064df3d] Failed to bin StandardFlowFileRecord[uuid=3e894a96-883a-4ac2-8121-b8200964cf20,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=6, container=in-memory, section=section], offset=0, length=5655],offset=0,name=b2c7cf61-b421-477d-902e-daeb2ed58f0d,size=5655] due to org.apache.nifi.controller.repository.ContentNotFoundException: Could not find content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=6, container=in-memory, section=section], offset=0, length=-1]: org.apache.nifi.controller.repository.ContentNotFoundException: Could not find content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=6, container=in-memory, section=section], offset=0, length=-1]
      org.apache.nifi.controller.repository.ContentNotFoundException: Could not find content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=6, container=in-memory, section=section], offset=0, length=-1]
      at org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)
      at org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)
      at org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)
      at org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)
      at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)
      at org.apache.nifi.processors.standard.MergeRecord.binFlowFile(MergeRecord.java:383)
      at org.apache.nifi.processors.standard.MergeRecord.onTrigger(MergeRecord.java:346)
      at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)
      at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
      at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
      at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

      QueryRecord:

      2021-07-06 10:15:09,174 ERROR [Timer-Driven Process Thread-4] o.a.nifi.processors.standard.QueryRecord QueryRecord[id=673fe9f6-017a-1000-8041-dfde9d02d976] Failed to determine Record Schema from StandardFlowFileRecord[uuid=090e3058-67e6-4436-bea9-d511132848e3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=2, container=in-memory, section=section], offset=0, length=5655],offset=0,name=090e3058-67e6-4436-bea9-d511132848e3,size=5655]; routing to failure: org.apache.nifi.controller.repository.ContentNotFoundException: Could not find content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=2, container=in-memory, section=section], offset=0, length=-1]
      org.apache.nifi.controller.repository.ContentNotFoundException: Could not find content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=2, container=in-memory, section=section], offset=0, length=-1]
      at org.apache.nifi.controller.repository.VolatileContentRepository.getContent(VolatileContentRepository.java:445)
      at org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:468)
      at org.apache.nifi.controller.repository.VolatileContentRepository.read(VolatileContentRepository.java:473)
      at org.apache.nifi.controller.repository.StandardProcessSession.getInputStream(StandardProcessSession.java:2302)
      at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2409)
      at org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:294)
      at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
      at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)
      at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
      at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
      at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

      SplitJson:

      2021-07-06 10:15:10,178 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.SplitJson SplitJson[id=7b411bdc-017a-1000-0f48-53d6a2ad5ee9] SplitJson[id=7b411bdc-017a-1000-0f48-53d6a2ad5ee9] failed to process session due to java.lang.NullPointerException; Processor Administratively Yielded for 1 sec: java.lang.NullPointerException
      java.lang.NullPointerException: null
      at org.apache.nifi.processors.standard.SplitJson.onTrigger(SplitJson.java:199)
      at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
      at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)
      at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
      at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
      at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

      This issue is not reproducible at 1.13.0 by my side, so it could correlate with the commit 528fce2407d092d4ced1a58fcc14d0bc6e660b89.

      With some support I would be glad to help investigate and solve the issue.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mathemare Matthieu RÉ

              Dates

              • Created:
                Updated:

                Issue deployment