Flume
  1. Flume
  2. FLUME-1779

Backoff returned from HDFSEventSink after IOException

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Status.Backoff is returned from HdfsEventSink.process() in case of IOException. This behavior prevents FailoverSinkProcessor from pushing event to next sink in queue.

      In my test case, IOException is caused by serious hdfs failure, for example all DataNodes in cluster are dead. After such failure BucketWriter throws IOException and becomes unavailable - it probably should be removed from sfWriters map.

      1. FLUME-1799.patch
        0.8 kB
        Connor Woodson
      2. st
        10 kB
        Jaroslaw Grabowski

        Activity

          People

          • Assignee:
            Unassigned
            Reporter:
            Jaroslaw Grabowski
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development