Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-252

Update Tail to get rid of races and truncation problems.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.9.0, 0.9.1
    • Fix Version/s: 0.9.2
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      The first tail implementation used buffered readers and file readers. This caused problems because the read call was blocking and couldn't shutdown properly.

      The second tail implementation was mroe closely based on gnu tail's C implementation but relies on RandomAccessFile. This version had problems with races (restarting from beginning FLUME-218) and truncation (new test in FLUME-218 patch by Eric Sammer).

      The new approach will likely use NIO and nonblocking IO to act cleanly, or possibly use a JNI based approach to get to unix system calls to get and follow file descriptors or inode numbers.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jmhsieh Jonathan Hsieh
                Reporter:
                jmhsieh Jonathan Hsieh
              • Votes:
                2 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: