[NIFI-3189] ConsumeKafka 0.9 and 0.10 can cause consumer rebalance when backpressure is engaged

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.2.0
• Assignee: Koji Kawamura
• Reporter: Joe Witt
• Created: 12/Dec/16 19:51
• Updated: 30/Mar/17 20:45
• Resolved: 30/Mar/17 20:45

Description

ConsumeKafka processors can alert to rebalance issues when backpressure is engaged on the output connection and is then freed up. This is because we're not doing anything with those consumers for a period of time and the kafka client detects this and initiates a rebalance. We should ensure that even when we cannot send more data due to back pressure that we at least have some sort of keep alive behavior with the kafka client. Or, if that isn't an option we should at least document the situation.

Issue Links

• links to: GitHub Pull Request #1527