Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-515

Log cleanup can close a file channel opnened by Log.read before the transfer completes

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0, 0.8.1
    • Fix Version/s: 0.8.1
    • Component/s: None
    • Labels:

      Description

      If consumers are lagging behind, then log cleanup activities can close a file channel opened by Log.read
      1. before the transfer the starts (broker will probably throw an exception in this case) OR
      2. during the transfer (possibility of half baked corrupted data being sent to consumer?)

      We probably haven't hit this race condition in practice because the consumers consume data well before the logs are cleaned up.

      To avoid this issue, we could avoid cleaning up the file until the transfer is complete. Reference counting?

        Attachments

          Activity

            People

            • Assignee:
              jkreps Jay Kreps
              Reporter:
              swapnilghike Swapnil Ghike
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: