Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.8.0
-
None
-
None
Description
Due to the watcher and zk data structure mismatch, consumer will not rebalance upon topic partition change. Details:
ZK data structure for topic partitions:
/brokers/topics/[topic-name]/partitions/[partition-id]/state
When new partitions are added, it will change the data in /brokers/topics/[topic-name], however the ZK client is watching on child change of /brokers/topics/[topic-name], which will always be a single child 'partitions'.
Proposal: add a data change listener, which will trigger rebalance upon data changes on /brokers/topics/[topic-name].