Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-1619

PutElasticSearch throws FlowFileHandlingException when id attribute is missing

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.5.1
    • Fix Version/s: 0.6.0
    • Component/s: Extensions
    • Labels:

      Description

      When the processor can't find the id_attribute on the FlowFile while trying to be added to the Bulk Request it is transferred to failure relationship. The issue is after this happens the FlowFile is still added to the Bulk Request even though a invalid id will be used during the insert. This will cause the a failure to be returned to the BulkResponse and a attempt to transfer the already transferred FlowFile throws a FlowFileHandlingException .

      Looks like a "continue;" added to line 178 would solve this issue.

      STACK:
      2016-03-11 07:49:39,053 ERROR [Timer-Driven Process Thread-12] o.a.n.p.elasticsearch.PutElasticsearch PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] No value in identifier attribute ${metadataId} for StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1457700558252-17754, container=default, section=346], offset=648997, length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]

      2016-03-11 07:49:39,081 ERROR [Timer-Driven Process Thread-12] o.a.n.p.elasticsearch.PutElasticsearch PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] Failed to insert into Elasticsearch due to StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1457700558252-17754, container=default, section=346], offset=648997, length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283] is already marked for transfer: org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1457700558252-17754, container=default, section=346], offset=648997, length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283] is already marked for transfer

      2016-03-11 07:49:39,105 ERROR [Timer-Driven Process Thread-12] o.a.n.p.elasticsearch.PutElasticsearch
      org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1457700558252-17754, container=default, section=346], offset=648997, length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283] is already marked for transfer
      at org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2369) ~[na:na]
      at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1782) ~[na:na]
      at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1777) ~[na:na]
      at org.apache.nifi.processors.elasticsearch.PutElasticsearch.onTrigger(PutElasticsearch.java:171) ~[nifi-elasticsearch-processors-0.5.1.jar:0.5.1]
      at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-0.5.1.jar:0.5.1]
      at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139) [nifi-framework-core-0.5.1.jar:0.5.1]
      at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) [nifi-framework-core-0.5.1.jar:0.5.1]
      at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) [nifi-framework-core-0.5.1.jar:0.5.1]
      at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124) [nifi-framework-core-0.5.1.jar:0.5.1]
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_95]
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_95]
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_95]
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_95]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_95]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_95]
      at java.lang.Thread.run(Thread.java:745) [na:1.7.0_95]

        Attachments

          Activity

            People

            • Assignee:
              mattyb149 Matt Burgess
              Reporter:
              vtmadcow Jim Wagoner
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: