Flume
  1. Flume
  2. FLUME-1238

Support active rolling of files created by HDFS Event Sink

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.1.0
    • Fix Version/s: v1.2.0
    • Component/s: None
    • Labels:
      None

      Description

      The HDFS Event Sink uses lazy rolling for closing files that are being written to. This results in many files being open for longer than their expected roll-interval if they are not actively written to and can even last in open state until the sink shuts down.

      It will be preferable to have these files roll proactively rather than waiting for a write to trigger the roll.

      1. FLUME-1238-2.patch
        30 kB
        Mike Percy
      2. FLUME-1238-1.patch
        29 kB
        Mike Percy

        Activity

          People

          • Assignee:
            Mike Percy
            Reporter:
            Arvind Prabhakar
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development