[CASSANDRA-15013] Prevent client requests from blocking on executor task queue

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.0.19, 3.11.5, 4.0-alpha1, 4.0
• Component/s: Messaging/Client
• Labels: pull-request-available
• Severity: Normal
• Impacts: Clients
• Since Version: 2.0.0

Source Control Link

5a03898c680ed6ada63901e8a4b278ccc8070717

Test and Documentation Plan

Passing UTs: https://circleci.com/gh/sumanth-pasupuleti/cassandra/288
Passing jvm dtests: https://circleci.com/gh/sumanth-pasupuleti/cassandra/287
DTests with 2 failures (seems unrelated): https://circleci.com/gh/sumanth-pasupuleti/cassandra/290#tests/containers/81, https://circleci.com/gh/sumanth-pasupuleti/cassandra/289#tests/containers/66

• Assignee: Sumanth Pasupuleti
• Reporter: Sumanth Pasupuleti

Authors

Sumanth Pasupuleti

Reviewers

Benedict Elliott Smith

• Created: 07/Feb/19 17:22
• Updated: 03/Sep/20 13:16
• Resolved: 15/Jul/19 13:55

Description

This is a follow-up ticket out of CASSANDRA-14855, to make the Flusher queue bounded, since, in the current state, items get added to the queue without any checks on queue size, nor with any checks on netty outbound buffer to check the isWritable state.
We are seeing this issue hit our production 3.0 clusters quite often.

Issue Links

• is related to: CASSANDRA-16100 Blog post on Improving Resiliency w/ "Prevent client requests from blocking on executor task queue"