[KAFKA-364] Consumer re-design

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Neha Narkhede
• Reporter: Neha Narkhede
• Created: 11/Jun/12 23:49
• Updated: 18/Apr/16 09:14
• Resolved: 18/Apr/16 09:14

Description

We've received quite a lot of feedback on the consumer side features over the past few months. Some of them are improvements to the current consumer design and some are simply new feature/API requests. I have attempted to write up the requirements that I've heard on this wiki -

https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Client+Re-Design

This would involve some significant changes to the consumer APIs, so we would like to collect feedback on the proposal from our community. Since the list of changes is not small, we would like to understand if some features are preferred over others, and more importantly, if some features are not required at all.

Issue Links

• is blocked by: KAFKA-264 Change the consumer side load balancing and distributed co-ordination to use a consumer co-ordinator
• is duplicated by: KAFKA-1326 New consumer checklist

Sub-Tasks

• Add ability to disable rebalancing in ZooKeeper consumer: Resolved
• Add ability to get offsets from ZK-based consumer: Resolved
• Expose offset vector to the consumer: Resolved
• Support for non-blocking polling on multiple streams: Resolved
• Support locality in consumer partition assignment algorithm: Resolved