[CASSANDRA-18555] Expose decommission state to nodetool info

• Type: Task
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 5.0-alpha1, 5.0
• Component/s: Observability/JMX
• Change Category: Operability
• Complexity: Normal
• Platform: All

Source Control Link

https://github.com/apache/cassandra/commit/e2a6c99310aa93ba3506ca8f603ae1039372f533

Test and Documentation Plan

added jvm dtest + ci

• Assignee: Jaydeepkumar Chovatia
• Reporter: Jaydeepkumar Chovatia

Authors

Jaydeepkumar Chovatia, Stefan Miklosovic

Reviewers

Brandon Williams, Maxwell Guo

• Created: 27/May/23 04:48
• Updated: 16/Nov/23 09:57
• Resolved: 20/Jun/23 20:40

Description

Currently, when a node is being decommissioned and if any failure happens, then an exception is thrown back to the caller.

But Cassandra's decommission takes considerable time ranging from minutes to hours to days. There are various scenarios in that the caller may need to probe the status again:

• The caller times out
• It is not possible to keep the caller hanging for such a long time

And If the caller does not know what happened internally, then it cannot retry, etc., leading to other issues.

So, in this ticket, I am going to add a new nodetool/JMX command that can be invoked by the caller anytime, and it will return the correct status.

It might look like a smaller change, but when we need to operate Cassandra at scale in a large-scale fleet, then this becomes a bottleneck and require constant operator intervention.

Issue Links

• relates to: CASSANDRA-18749 Expose bootstrap failure state via JMX
• links to: GitHub Pull Request #2390