Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-682

Improve SeqfileEventSink performance

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.4
    • Fix Version/s: 0.9.5
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      The SeqfileEventSink disables output buffering in exchange for durability, however due to the async relationship between the source and sink, no such guarantees are possible. When using the SyslogTcpSourceThreads which enlists an internal buffer to store messages before they are read, a server crash would result in all buffered messages being lost. The lack of output buffering makes this much more likely to happen.

      See previous discussion on flume-dev list: http://bit.ly/jQ1Gsv

        Attachments

          Activity

            People

            • Assignee:
              flume_chetan Disabled imported user
              Reporter:
              flume_csarva Disabled imported user
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: