Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-993

Tail thread nterrupte: null / Tail thread exiting With S3 bucket

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Won't Fix
    • Affects Version/s: 0.9.4
    • Fix Version/s: 0.9.5
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Environment:

      Debian Squeeze, S3, EC2

      Description

      Hi Guys,
      The following are logs from my flume agents. These agents are using a collector sink that pushes to s3. I am finding that my agents will not stay alive and get into the blow error state with the tail thread exiting. Please advise. Thanks much!

      2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
      2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
      @
      9942,93 22%
      2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
      java.lang.InterruptedException
      at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
      at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
      at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
      at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
      at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
      2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
      2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
      2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
      2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-05144
      26931-0800.316601632689443.00000026
      2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_ww
      riting/20120222-051426931-0800.316601632689443.00000026
      2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.31660163266
      89443.00000026
      2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaa
      iloverManager$StateChangeDeco@65cccd3
      2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
      2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
      2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData:

      {srcVer:'Wed Feb 22 05:14:34 PSTT 2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' ss ink:'agentSink( "flume-collector", 35853 )' }

      2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
      2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
      @
      9942,93 22%
      2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
      java.lang.InterruptedException
      at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
      at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
      at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
      at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
      at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
      2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
      2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
      2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
      2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-051422
      6931-0800.316601632689443.00000026
      2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrr
      iting/20120222-051426931-0800.316601632689443.00000026
      2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.316601632688
      9443.00000026
      2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaii
      loverManager$StateChangeDeco@65cccd3
      2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
      2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
      2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData:

      {srcVer:'Wed Feb 22 05:14:34 PST 2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sinn k:'agentSink( "flume-collector", 35853 )' }

      2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
      2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
      @
      9942,93 22%
      2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
      java.lang.InterruptedException
      at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
      at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
      at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
      at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
      at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
      2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
      2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
      2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
      2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-0514266
      931-0800.316601632689443.00000026
      2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrii
      ting/20120222-051426931-0800.316601632689443.00000026
      2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.3166016326899
      443.00000026
      2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaill
      overManager$StateChangeDeco@65cccd3
      2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
      2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
      2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData:

      {srcVer:'Wed Feb 22 05:14:34 PST 22 012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sink:: 'agentSink( "flume-collector", 35853 )' }

      2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
      2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
      @
      9942,93 22%
      2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
      java.lang.InterruptedException
      at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
      at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
      at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
      at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
      at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
      2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              zenloop@gmail.com Julian Alcala
            • Votes:
              2 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: