Major Description
Currently the consumer will block in poll() after receiving its assignment in order to set the starting offset for every assigned partition. If the topic is deleted or if a partition is unavailable, the consumer can be stuck indefinitely. Most of the time this is not a problem since the starting offset is obtained from the committed offsets, which does not depend on partition availability. However, if there are no committed offsets or if the user has manually called seekToBeginning or seekToEnd, then we will need to do a lookup for the starting offset from the partition leader, which will stall the consumer until the partition is available or recreated. It would be better to let the consumer fetch on partitions which are available and periodically check availability for the rest.
Attachments
Issue Links
- links to