[CASSANDRA-16850] Add client warnings and abort to tombstone and coordinator reads which go past a low/high watermark

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.1-alpha1, 4.1
• Component/s: Observability/Logging
• Change Category: Operability
• Complexity: Normal
• Platform: All

Source Control Link

https://github.com/apache/cassandra/commit/9cbad6007c008507dfd0a7a7dd92429d712e669f

Test and Documentation Plan

client / unit testing

• Assignee: David Capwell
• Reporter: David Capwell

Authors

David Capwell

Reviewers

Blake Eggleston, Marcus Eriksson

• Created: 12/Aug/21 18:46
• Updated: 27/May/22 19:24
• Resolved: 26/Aug/21 20:41

Description

We currently will abort queries if we hit too many tombstones, but its common that we would want to also warn clients (client warnings) about this before we get that point; its also common that different logic would like to be able to warn/abort about client options (such as reading a large partition). To allow this we should add a concept of low/high watermarks (warn/abort) to tombstones and coordinator reads.

Another issue is that current aborts look the same as a random failure, so from an SLA point of view it would be good to differentiate between user behavior being rejected and unexplained issue.

Issue Links

• links to: Circle CI: trunk
• links to: GH PR: trunk