[KAFKA-766] Isr shrink/expand check is fragile

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9.0.0
• Assignee: Neha Narkhede
• Reporter: Sriram
• Created: 20/Feb/13 19:05
• Updated: 08/Apr/15 21:06
• Resolved: 08/Apr/15 21:06

Description

Currently the isr check is coupled tightly with the produce batch size. For example, if the producer batch size is 10000 messages and isr check is 4000 messages, we continuously oscillate between shrinking isr and expanding isr every second. This is because a single produce request throws the replica out of the isr. This results in hundreds of calls to ZK (we still dont have multi write). This can be alleviated by making the producer batch size smaller than the isr check size.

Going forward, we should try to not have this coupling. It is worth investigating if we can make the check more robust under such scenarios.