Minor Description
After KAFKA-657, we have a protocol for consumers to commit and fetch offsets from brokers. Currently, consumers are not using this API and directly talking with Zookeeper.
This Jira will involve following:
1. Add a special topic in kafka for storing offsets
2. Add an OffsetManager interface which would handle storing, accessing, loading and maintaining consumer offsets
3. Implement offset managers for both of these 2 choices : existing ZK based storage or inbuilt storage for offsets.
4. Leader brokers would now maintain an additional hash table of offsets for the group-topic-partitions that they lead
5. Consumers should now use the OffsetCommit and OffsetFetch API