Kafka
  1. Kafka
  2. KAFKA-116

AsyncProducer shutdown logic causes data loss

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7
    • Fix Version/s: 0.7
    • Component/s: None
    • Labels:
      None

      Description

      The current shutdown logic of the AsyncProducer allows adding events to the queue, after adding the shutdown command to it. The ProducerSendThread drains all the data in the queue until it hits the shutdown command. Hence, all data added after the shutdown command is lost.

      1. KAFKA-116.patch
        10 kB
        Neha Narkhede

        Activity

        Neha Narkhede created issue -
        Hide
        Neha Narkhede added a comment -

        1. Fixed the shutdown bug by setting the closed queue flag before adding the shutdown command to the async producer queue
        2. Added more logging at the trace level
        3. Refactored the blocking async producer logic to use case match instead of nested if else

        Show
        Neha Narkhede added a comment - 1. Fixed the shutdown bug by setting the closed queue flag before adding the shutdown command to the async producer queue 2. Added more logging at the trace level 3. Refactored the blocking async producer logic to use case match instead of nested if else
        Neha Narkhede made changes -
        Field Original Value New Value
        Attachment KAFKA-116.patch [ 12491322 ]
        Neha Narkhede made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Jun Rao added a comment -

        +1

        Show
        Jun Rao added a comment - +1
        Hide
        Chris Burroughs added a comment -

        IllegalQueueStateException was missed in the commit.

        Show
        Chris Burroughs added a comment - IllegalQueueStateException was missed in the commit.
        Hide
        Neha Narkhede added a comment -

        thanks for pointing that out. I checked it in now.

        Show
        Neha Narkhede added a comment - thanks for pointing that out. I checked it in now.
        Jun Rao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.7 [ 12317243 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Neha Narkhede
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development