Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1779

Backoff returned from HDFSEventSink after IOException

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Status.Backoff is returned from HdfsEventSink.process() in case of IOException. This behavior prevents FailoverSinkProcessor from pushing event to next sink in queue.

      In my test case, IOException is caused by serious hdfs failure, for example all DataNodes in cluster are dead. After such failure BucketWriter throws IOException and becomes unavailable - it probably should be removed from sfWriters map.

        Attachments

        1. FLUME-1799.patch
          0.8 kB
          Connor Woodson
        2. st
          10 kB
          Jaroslaw Grabowski

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jtgrabowski Jaroslaw Grabowski
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: