Kafka
  1. Kafka
  2. KAFKA-1075

Consumer will not rebalance upon topic partition change

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      Due to the watcher and zk data structure mismatch, consumer will not rebalance upon topic partition change. Details:

      ZK data structure for topic partitions:

      /brokers/topics/[topic-name]/partitions/[partition-id]/state

      When new partitions are added, it will change the data in /brokers/topics/[topic-name], however the ZK client is watching on child change of /brokers/topics/[topic-name], which will always be a single child 'partitions'.

      Proposal: add a data change listener, which will trigger rebalance upon data changes on /brokers/topics/[topic-name].

      1. KAFKA-1075.patch
        4 kB
        Guozhang Wang
      2. KAFKA-1075_2013-10-07_10:25:01.patch
        20 kB
        Guozhang Wang
      3. KAFKA-1075_2013-10-07_10:58:40.patch
        4 kB
        Guozhang Wang

        Activity

          People

          • Assignee:
            Guozhang Wang
            Reporter:
            Guozhang Wang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development