Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-723

hello-samza hangs when we use StreamAppender

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.10.0
    • 0.10.0
    • hello-samza
    • None

    Description

      I added StreamAppender to log4j.xml , task.log4j.system=kafka in the config and added a simple log line in process call to see if it creates the logging event correctly.
      When I deploy the job (WikipediaFeedStreamTask.java), the AppMaster seems to just hang in the bootstrap method. I don’t get any exceptions in the AppMaster logs. I checked the kafka logs and found some socket closed errors in the stack trace.

      ==> deploy/kafka/logs/kafka.log <==
      [2015-06-25 16:48:01,179] ERROR Closing socket for /172.21.132.57 because of error (kafka.network.Processor)
      java.io.IOException: Connection reset by peer
      at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
      at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
      at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
      at sun.nio.ch.IOUtil.write(IOUtil.java:65)
      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
      at kafka.api.TopicDataSend.writeTo(FetchResponse.scala:123)
      at kafka.network.MultiSend.writeTo(Transmission.scala:101)
      at kafka.api.FetchResponseSend.writeTo(FetchResponse.scala:231)
      at kafka.network.Processor.write(SocketServer.scala:472)
      at kafka.network.Processor.run(SocketServer.scala:342)
      at java.lang.Thread.run(Thread.java:745)

      ==> deploy/kafka/logs/server.log <==
      [2015-06-25 16:48:01,179] ERROR Closing socket for /172.21.132.57 because of error (kafka.network.Processor)
      java.io.IOException: Connection reset by peer
      at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
      at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
      at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
      at sun.nio.ch.IOUtil.write(IOUtil.java:65)
      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
      at kafka.api.TopicDataSend.writeTo(FetchResponse.scala:123)
      at kafka.network.MultiSend.writeTo(Transmission.scala:101)
      at kafka.api.FetchResponseSend.writeTo(FetchResponse.scala:231)
      at kafka.network.Processor.write(SocketServer.scala:472)
      at kafka.network.Processor.run(SocketServer.scala:342)
      at java.lang.Thread.run(Thread.java:745)

      Need to investigate this issue further. looks like the CoordinatorStream is conflicting with the StreamAppender components.

      Attachments

        1. SAMZA-723-2.patch
          13 kB
          Navina Ramesh
        2. SAMZA-723.1.patch
          15 kB
          Yan Fang
        3. SAMZA-723-asynAppender.patch
          2 kB
          Yan Fang

        Issue Links

          Activity

            People

              closeuris Yan Fang
              navina Navina Ramesh
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: