Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-723

hello-samza hangs when we use StreamAppender

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.10.0
    • Component/s: hello-samza
    • Labels:
      None

      Description

      I added StreamAppender to log4j.xml , task.log4j.system=kafka in the config and added a simple log line in process call to see if it creates the logging event correctly.
      When I deploy the job (WikipediaFeedStreamTask.java), the AppMaster seems to just hang in the bootstrap method. I don’t get any exceptions in the AppMaster logs. I checked the kafka logs and found some socket closed errors in the stack trace.

      ==> deploy/kafka/logs/kafka.log <==
      [2015-06-25 16:48:01,179] ERROR Closing socket for /172.21.132.57 because of error (kafka.network.Processor)
      java.io.IOException: Connection reset by peer
      at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
      at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
      at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
      at sun.nio.ch.IOUtil.write(IOUtil.java:65)
      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
      at kafka.api.TopicDataSend.writeTo(FetchResponse.scala:123)
      at kafka.network.MultiSend.writeTo(Transmission.scala:101)
      at kafka.api.FetchResponseSend.writeTo(FetchResponse.scala:231)
      at kafka.network.Processor.write(SocketServer.scala:472)
      at kafka.network.Processor.run(SocketServer.scala:342)
      at java.lang.Thread.run(Thread.java:745)

      ==> deploy/kafka/logs/server.log <==
      [2015-06-25 16:48:01,179] ERROR Closing socket for /172.21.132.57 because of error (kafka.network.Processor)
      java.io.IOException: Connection reset by peer
      at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
      at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
      at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
      at sun.nio.ch.IOUtil.write(IOUtil.java:65)
      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
      at kafka.api.TopicDataSend.writeTo(FetchResponse.scala:123)
      at kafka.network.MultiSend.writeTo(Transmission.scala:101)
      at kafka.api.FetchResponseSend.writeTo(FetchResponse.scala:231)
      at kafka.network.Processor.write(SocketServer.scala:472)
      at kafka.network.Processor.run(SocketServer.scala:342)
      at java.lang.Thread.run(Thread.java:745)

      Need to investigate this issue further. looks like the CoordinatorStream is conflicting with the StreamAppender components.

        Attachments

        1. SAMZA-723-asynAppender.patch
          2 kB
          Yan Fang
        2. SAMZA-723-2.patch
          13 kB
          Navina Ramesh
        3. SAMZA-723.1.patch
          15 kB
          Yan Fang

          Issue Links

            Activity

              People

              • Assignee:
                closeuris Yan Fang
                Reporter:
                navina Navina Ramesh
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: