Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2320

Deadlock in DatasetSink

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.0
    • 1.5.0
    • Sinks+Sources
    • None

    Description

      Lines 251-252 in the DatasetSink are a potential deadlock: if the transaction throws an exception, then the writer lock is not released, but the same thread tries to lock in the error handling.

      While the simplest solution is to move those two lines inside the try/finally statement, I think we can actually remove the lock completely by reverting to the original version that rolled the files in the process() method. The original concern about that design was that there needs to be some guarantee that all files will be rolled. Because the SinkRunner has a max backoff, there is a guaranteed maximum amount of time between calls to process.

      Attachments

        1. FLUME-2320-1.patch
          11 kB
          Ryan Blue
        2. 0002-FLUME-2320-Fix-deadlock-by-removing-writer-lock.patch
          11 kB
          Ryan Blue
        3. FLUME-2320-2.patch
          11 kB
          Ryan Blue

        Activity

          People

            rdblue Ryan Blue
            rdblue Ryan Blue
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: