Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-12920

Consumer's cooperative sticky assignor need to clear generation / assignment data upon `onPartitionsLost`

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Bug
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Consumer's cooperative-sticky assignor does not track the owned partitions inside the assignor — i.e. when it reset its state in event of ``onPartitionsLost``, the ``memberAssignment`` and ``generation`` inside the assignor would not be cleared. This would cause a member to join with empty generation on the protocol while with non-empty user-data encoding the old assignment still (and hence pass the validation check on broker side during JoinGroup), and eventually cause a single partition to be assigned to multiple consumers within a generation.

      We should let the assignor to also clear its assignment/generation when ``onPartitionsLost`` is triggered in order to avoid this scenario.

      Note that 1) for the regular sticky assignor the generation would still have an older value, and this would cause the previously owned partitions to be discarded during the assignment, and 2) for Streams' sticky assignor, it’s encoding would indeed be cleared along with ``onPartitionsLost``. Hence only Consumer's cooperative-sticky assignor have this issue to solve.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                guozhang Guozhang Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: