[KAFKA-4060] Remove ZkClient dependency in Kafka Streams

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.2.0
• Component/s: streams
• Labels: kip
• Assignee: Hojjat Jafarpour
• Reporter: Guozhang Wang
• Reviewer: Guozhang Wang
• Created: 17/Aug/16 22:34
• Updated: 23/Jan/17 18:21
• Resolved: 11/Jan/17 17:15

Description

In Kafka Streams we need to dynamically create or update those internal topics (i.e. repartition topics) upon rebalance, inside InternalTopicManager which is triggered by StreamPartitionAssignor. Currently we are using ZkClient to talk to ZK directly for such actions.

With create and delete topics request merged in by granthenke as part of KIP-4, we should now be able to remove the ZkClient dependency and directly use these requests.

Related:
1. KIP-4. https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations

2. Consumer Reblance Protocol. https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Client-side+Assignment+Proposal

Issue Links

• relates to: KAFKA-4369 ZkClient is not closed upon streams shutdown
• links to: GitHub Pull Request #1884
• links to: GitHub Pull Request #2375
• links to: GitHub Pull Request #2377
• links to: GitHub Pull Request #2404
• links to: GitHub Pull Request #2418
• mentioned in: Page Loading...