[CASSANDRA-12231] Make large mutations easier to find

• Type: Improvement
• Status: Open
• Priority: Low
• Resolution: Unresolved
• Component/s: Legacy/Observability
• Assignee: Unassigned
• Reporter: Ryan Svihla
• Created: 19/Jul/16 02:47
• Updated: 16/Apr/19 09:30

Description

Apologies if this has already been submitted, my lacking Jira search foo will be to blame.

There are two problems related to large mutations:
1. Mutations that are really large and fail the write with the following "Mutation of %s is too large for the maximum size of %s" (in CommitLog.java). While this should be something the clients can handle, they often dont and the DBA is the first person to notice this. If we could log the primary key attempted it could be helpful to track down the culprit.
2. Mutations that are still too large but still under the threshold, so they silently crush the server. We could add a warn to this the same way we do with batch logs with ideally a configurable threshold. It would also be handy if we could include the primary key used. Today, these are super nasty today to track down and involve a scan of the dataset using something like Spark.

Issue Links

• relates to: CASSANDRA-14781 Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough