Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-12920

Consumer's cooperative sticky assignor need to clear generation / assignment data upon `onPartitionsLost`

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Bug
    • None
    • None
    • None

    Description

      Consumer's cooperative-sticky assignor does not track the owned partitions inside the assignor — i.e. when it reset its state in event of ``onPartitionsLost``, the ``memberAssignment`` and ``generation`` inside the assignor would not be cleared. This would cause a member to join with empty generation on the protocol while with non-empty user-data encoding the old assignment still (and hence pass the validation check on broker side during JoinGroup), and eventually cause a single partition to be assigned to multiple consumers within a generation.

      We should let the assignor to also clear its assignment/generation when ``onPartitionsLost`` is triggered in order to avoid this scenario.

      Note that 1) for the regular sticky assignor the generation would still have an older value, and this would cause the previously owned partitions to be discarded during the assignment, and 2) for Streams' sticky assignor, it’s encoding would indeed be cleared along with ``onPartitionsLost``. Hence only Consumer's cooperative-sticky assignor have this issue to solve.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              guozhang Guozhang Wang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: