[CASSANDRA-7402] Add metrics to track memory used by client requests

• Type: Improvement
• Status: Open
• Priority: Low
• Resolution: Unresolved
• Fix Version/s: 5.x
• Component/s: Observability/Metrics
• Labels: ops performance stability
• Assignee: T Jake Luciani Assign to me
• Reporter: T Jake Luciani

Authors

T Jake Luciani

• Created: 16/Jun/14 14:00
• Updated: 07/Mar/23 10:54

Description

When running a production cluster one common operational issue is quantifying GC pauses caused by ongoing requests.

Since different queries return varying amount of data you can easily get your self into a situation where you Stop the world from a couple of bad actors in the system. Or more likely the aggregate garbage generated on a single node across all in flight requests causes a GC.

It would be very useful for operators to see how much garbage the system is using to handle in flight mutations and queries.

It would also be nice to have either a log of queries which generate the most garbage so operators can track this. Also a histogram.

Issue Links

• is related to: CASSANDRA-3017 add a Message size limit
• relates to: CASSANDRA-9318 Bound the number of in-flight requests at the coordinator