Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6760

Load-Balanced Connections can error if configured to compress content

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.10.0
    • Component/s: Core Framework
    • Labels:
      None

      Description

      Using load-balanced connections, I can sometimes see failures when data is being compressed. On the client side, I see in the logs:

      2019-10-09 15:19:40,688 ERROR [Load-Balanced Client Thread-4] o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Failed to communicate with Peer localhost:5672
      java.io.IOException: Broken pipe
         at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
         at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
         at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
         at sun.nio.ch.IOUtil.write(IOUtil.java:65)
         at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
         at org.apache.nifi.controller.queue.clustered.client.async.nio.PeerChannel.write(PeerChannel.java:132)
         at org.apache.nifi.controller.queue.clustered.client.async.nio.LoadBalanceSession.communicate(LoadBalanceSession.java:165)
         at org.apache.nifi.controller.queue.clustered.client.async.nio.NioAsyncLoadBalanceClient.communicate(NioAsyncLoadBalanceClient.java:242)
         at org.apache.nifi.controller.queue.clustered.client.async.nio.NioAsyncLoadBalanceClientTask.run(NioAsyncLoadBalanceClientTask.java:76)
         at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
         at java.lang.Thread.run(Thread.java:745)
      
      

      While on the server side I see:

      2019-10-09 15:19:40,687 ERROR [Load-Balance Server Thread-4] o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with Peer /127.0.0.1:50895
      java.io.EOFException: Unexpected end of ZLIB input stream
         at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
         at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
         at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
         at org.apache.nifi.stream.io.StreamUtils.fillBuffer(StreamUtils.java:89)
         at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.consumeContent(StandardLoadBalanceProtocol.java:512)
         at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFile(StandardLoadBalanceProtocol.java:445)
         at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:252)
         at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:133)
         at org.apache.nifi.controller.queue.clustered.server.ConnectionLoadBalanceServer$CommunicateAction.run(ConnectionLoadBalanceServer.java:152)
         at java.lang.Thread.run(Thread.java:745)
       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                markap14 Mark Payne
                Reporter:
                markap14 Mark Payne
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 20m
                  2h 20m