[CASSANDRA-7545] Hints for a down node are written to a single partition in system.hints on the coordinator leading to contention

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Duplicate
• Severity: Normal
• Assignee: Unassigned
• Reporter: graham sanderson
• Created: 15/Jul/14 00:50
• Updated: 16/Apr/19 09:31
• Resolved: 19/May/15 17:14

Description

The worst side effect is potentially orders of magnitude larger than suspected memory allocation due to a race condition updating the columns in the memtable (see CASSANDRA-7546)

That said, having so many hints in a single partition has other negative side effects (non linear growth of sorted tree data structure overhead in memtable, just plain large number of tombstones in a row etc.)

Ideally, the hints would be partitioned by both the nodeUUID, and a fixed number of (say 4) bits of a hash of the original rowmutation's partition key... even this small spreading of load has a significant improvement.

Issue Links

• duplicates: CASSANDRA-6230 Write hints to flat files instead of the system.hints
• relates to: CASSANDRA-7546 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory