Jun's comments -
1, 4, 5, 6. Fixed it
2. Changed the ZkQueue to be sort of a blocking queue as per Jay's suggestion. It is not a complete blocking queue since the put operation does not block. The leader needs to know that the queue is full immediately to be able to initiate some kind of queue shrink operation. The dequeue operation is blocking.
3.1 Removed the StateChangeListener since now, the ZkQueue wraps its own listener
3.2 Good point. Fixed it.
Jay's comments -
1. You have a valid point here. I agree that we should be able to refactor the code to wrap up custom ZK logic in Kafka classes. Right now, the leader election stuff it stuck in there and hence the epoch increment API too. But since we might end up changing the leader election algorithm itself, I would suggest waiting a bit before attempting this refactoring.
2. Changed ZkQueue and StateChangeCommand for consistency.
3. I like this suggestion, gave it a shot. Let me know what you think.