Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2357

HDFS sink should retry closing files that previously had close errors

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.5.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      When the AbstractHDFSWriter fails to close a file (due to exceeding the callTimeout or other hdfs issues), it will leave the the file open and never try again. The only way to close the open files is to restart the flume agent. There should be a configurable option to allow the sink to retry to close files that had previously failed to close.

        Attachments

        1. FLUME-2357-3.patch
          62 kB
          Hari Shreedharan
        2. FLUME-2357-2.patch
          62 kB
          Hari Shreedharan
        3. FLUME-2357-1.patch
          62 kB
          Hari Shreedharan
        4. FLUME-2357.patch
          55 kB
          Hari Shreedharan

          Issue Links

            Activity

              People

              • Assignee:
                hshreedharan Hari Shreedharan
                Reporter:
                pdvorak Patrick Dvorak
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: