Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-515

Log cleanup can close a file channel opnened by Log.read before the transfer completes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.8.0, 0.8.1
    • 0.8.1
    • None

    Description

      If consumers are lagging behind, then log cleanup activities can close a file channel opened by Log.read
      1. before the transfer the starts (broker will probably throw an exception in this case) OR
      2. during the transfer (possibility of half baked corrupted data being sent to consumer?)

      We probably haven't hit this race condition in practice because the consumers consume data well before the logs are cleaned up.

      To avoid this issue, we could avoid cleaning up the file until the transfer is complete. Reference counting?

      Attachments

        Activity

          People

            jkreps Jay Kreps
            swapnilghike Swapnil Ghike
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: