Uploaded image for project: 'Commons IO'
  1. Commons IO
  2. IO-278

Improve Tailer performance with buffered reads

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.1
    • Fix Version/s: 2.4
    • Component/s: None
    • Labels:
      None

      Description

      I noticed Tailer read performances are pretty poor when dealing with large, frequently written, log files, and this is due to the use of RandomAccessFile which does unbuffered reads, hence causing lots of disk I/O.

      So I improved the Tailer implementation by introducing buffered reads: it works by loading large (configurable) file chunks in memory, and reading lines from there; this enhances performances in my tests from 10x to 30x depending on the file size.
      I also added two test cases: one to simulate reading of a large file (you can use it to compare performances), the other to verify correct handling on buffer breaks; obviously, all tests pass.

      I'm attaching the diff files, let me know if it's okay for you guys!

        Attachments

        1. TailerTest.diff
          7 kB
          Sergio Bossa
        2. Tailer.diff
          10 kB
          Sergio Bossa

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                sbtourist Sergio Bossa
              • Votes:
                3 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: