Flume
  1. Flume
  2. FLUME-1175

RollingFileSink complains of Bad File Descriptor upon a reconfig event

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: v1.2.0
    • Fix Version/s: v1.4.0
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Environment:

      CentOS 6.2 64-bit

      Description

      Steps:
      1) Create a config file that looks something like this:
      agent.channels = c1
      agent.sources = r1
      agent.sinks = k1
      #
      agent.channels.c1.type = MEMORY
      #
      agent.sources.r1.channels = c1
      agent.sources.r1.type = SEQ
      #
      agent.sinks.k1.channel = c1
      agent.sinks.k1.type = FILE_ROLL
      agent.sinks.k1.sink.directory = /var/log/flume-ng
      agent.sinks.k1.sink.rollInterval = 0

      2) Start the Flume NG agent

      3) touch the config file so that a reconfig event is triggered within 30 secs

      4) tail the output file to observer the sequence generator events:
      tail -f /var/log/flume-ng/XXXXXXXXXXXX

      5) Notice that the flow suddenly stops at the reconfig event (within 30 secs after touching the config file). Flow doesn't continue. The flume log shows a Bad File Descriptor error for the RollingFileSink:

      2012-05-03 01:34:34,806 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable to deliver event. Exception follows.
      org.apache.flume.EventDeliveryException: Failed to process event: [Event headers =

      {timestamp=1336034074797, nanos=3762297996593382, pri=INFO, host=<mysupersecrethost>, FlumeOG=yes, execcmd=java.nio.HeapByteBuffer[pos=0 lim=24 cap=24], procsource=java.nio.HeapByteBuffer[pos=0 lim=6 cap=6], service=java.nio.HeapByteBuffer[pos=0 lim=4 cap=4]}

      , body.length = 26 ]
      at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:201)
      at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
      at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
      at java.lang.Thread.run(Thread.java:662)
      Caused by: java.io.IOException: Bad file descriptor
      at java.io.FileOutputStream.writeBytes(Native Method)
      at java.io.FileOutputStream.write(FileOutputStream.java:282)
      at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
      at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
      at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:193)
      ... 3 more

        Issue Links

          Activity

            People

            • Assignee:
              Roshan Naik
              Reporter:
              Will McQueen
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development