Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.5.0, 1.11.3
-
None
-
Apache NiFi - Version 1.5.0.3.1.1.0-35
-
Important
Description
Description
A POST to an API that returns an empty body on success (status code 200) will generate an EOF Exception, causing the processor to always remain in error and routing all flow files to failures, even if the API call succeeded.
An example of such API is the Streaming API of PowerBI: https://docs.microsoft.com/en-us/power-bi/service-real-time-streaming
Exception Stack Traces
When the property "Put Response Body In Attribute" is not set, the exception is as follows:
rocessor.exception.FlowFileAccessException: Unable to create ContentClaim due to java.io.EOFException: org.apache.nifi.pro cessor.exception.FlowFileAccessException: Failed to import data from buffer(okio.GzipSource@159311b9).inputStream() for St andardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-position-20180313-182039.p b,size=0] due to org.apache.nifi.processor.exception.FlowFileAccessException: Unable to create ContentClaim due to java.io .EOFException org.apache.nifi.processor.exception.FlowFileAccessException: Failed to import data from buffer(okio.GzipSource@159311b9).i nputStream() for StandardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-positio n-20180313-182039.pb,size=0] due to org.apache.nifi.processor.exception.FlowFileAccessException: Unable to create ContentC laim due to java.io.EOFException at org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2942) at org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:817) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.nifi.processor.exception.FlowFileAccessException: Unable to create ContentClaim due to java.io.EOFException at org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2935) ... 13 common frames omitted Caused by: java.io.EOFException: null at okio.RealBufferedSource.require(RealBufferedSource.java:59) at okio.GzipSource.consumeHeader(GzipSource.java:114) at okio.GzipSource.read(GzipSource.java:73) at okio.RealBufferedSource$1.read(RealBufferedSource.java:409) at java.io.InputStream.read(InputStream.java:101) at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35) at org.apache.nifi.controller.repository.FileSystemRepository.importFrom(FileSystemRepository.java:734) at org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2932) ... 13 common frames omitted
When "Put Body Response in Attribute" property is set, the exception is as follows :
2018-03-13 18:32:13,165 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=5aae3a34-35d9-1268-6e6d-b774d4467e5a] Routing to Failure due to exception: java.io.EOFException: java.io.EOFException java.io.EOFException: null at okio.RealBufferedSource.require(RealBufferedSource.java:59) at okio.GzipSource.consumeHeader(GzipSource.java:114) at okio.GzipSource.read(GzipSource.java:73) at okio.RealBufferedSource$1.read(RealBufferedSource.java:409) at org.apache.nifi.stream.io.StreamUtils.fillBuffer(StreamUtils.java:89) at org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:844) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Possible Solution
- Wrap calls to fillBuffer and importFrom into a try-catch
- Catch the EOF exception
- Generate an empty response (empty attribute or empty flow file) when this case occurs
Alternative Solution
- Add a property indicating to the processor to ignore the body response
Additional Notes
- There is currently an attempt to handle such case in the code, but it is not sufficient.
boolean bodyExists = responseBody != null;
- The variable bodyExists is true in the above case, even if no body has been returned by the REST API.
