[NIFI-6760] Load-Balanced Connections can error if configured to compress content

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.10.0
• Component/s: Core Framework
• Assignee: Mark Payne
• Reporter: Mark Payne
• Created: 09/Oct/19 19:40
• Updated: 10/Oct/19 14:19
• Resolved: 10/Oct/19 14:19

Description

Using load-balanced connections, I can sometimes see failures when data is being compressed. On the client side, I see in the logs:

2019-10-09 15:19:40,688 ERROR [Load-Balanced Client Thread-4] o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Failed to communicate with Peer localhost:5672
java.io.IOException: Broken pipe
   at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
   at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
   at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
   at sun.nio.ch.IOUtil.write(IOUtil.java:65)
   at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
   at org.apache.nifi.controller.queue.clustered.client.async.nio.PeerChannel.write(PeerChannel.java:132)
   at org.apache.nifi.controller.queue.clustered.client.async.nio.LoadBalanceSession.communicate(LoadBalanceSession.java:165)
   at org.apache.nifi.controller.queue.clustered.client.async.nio.NioAsyncLoadBalanceClient.communicate(NioAsyncLoadBalanceClient.java:242)
   at org.apache.nifi.controller.queue.clustered.client.async.nio.NioAsyncLoadBalanceClientTask.run(NioAsyncLoadBalanceClientTask.java:76)
   at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)

While on the server side I see:

2019-10-09 15:19:40,687 ERROR [Load-Balance Server Thread-4] o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with Peer /127.0.0.1:50895
java.io.EOFException: Unexpected end of ZLIB input stream
   at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
   at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
   at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
   at org.apache.nifi.stream.io.StreamUtils.fillBuffer(StreamUtils.java:89)
   at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.consumeContent(StandardLoadBalanceProtocol.java:512)
   at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFile(StandardLoadBalanceProtocol.java:445)
   at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:252)
   at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:133)
   at org.apache.nifi.controller.queue.clustered.server.ConnectionLoadBalanceServer$CommunicateAction.run(ConnectionLoadBalanceServer.java:152)
   at java.lang.Thread.run(Thread.java:745)
 

 

Issue Links

• links to: GitHub Pull Request #3801