[SAMZA-2633] Rolling deployment/upgrade causes downtime for processors for the entire deployment window

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7
• Assignee: Bharath Kumarasubramanian
• Reporter: Bharath Kumarasubramanian
• Created: 13/Mar/21 01:29
• Updated: 06/May/21 00:04
• Resolved: 06/May/21 00:04

Description

Problem:

At LinkedIn, we noticed several standalone users complained about lag/downtime during rolling deployments/upgrades.

Description:

During rolling upgrades, the current debounce timer gets extended every time when there is a quorum change notification. As a result, processors that were upgraded earlier in the deployment window remain unavailable waiting for work assignment. In some scenarios, this cause processors to be unavailable for 20 minutes or so depending on the size of the quorum and the debounce time configuration.

Impact:

Partitions that were stopped for initial processors as part of upgrade remain unassigned for the entire deployment window which can result in processing lag.

Issue Links

• links to: Standalone rolling upgrade issue