Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15367

Memtable memory allocations may deadlock

    XMLWordPrintableJSON

    Details

    • Bug Category:
      Availability - Process Crash
    • Severity:
      Critical
    • Complexity:
      Challenging
    • Discovered By:
      User Report
    • Platform:
      All
    • Impacts:
      None
    • Test and Documentation Plan:
      Hide

      unit test included

      Show
      unit test included

      Description

      • Under heavy contention, we guard modifications to a partition with a mutex, for the lifetime of the memtable.
      • Memtables block for the completion of all OpOrder.Group started before their flush began
      • Memtables permit operations from this cohort to fall-through to the following Memtable, in order to guarantee a precise commitLogUpperBound
      • Memtable memory limits may be lifted for operations in the first cohort, since they block flush (and hence block future memory allocation)

      With very unfortunate scheduling

      • A contended partition may rapidly escalate to a mutex
      • The system may reach memory limits that prevent allocations for the new Memtable’s cohort (C2)
      • An operation from C2 may hold the mutex when this occurs
      • Operations from a prior Memtable’s cohort (C1), for a contended partition, may fall-through to the next Memtable
      • The operations from C1 may execute after the above is encountered by those from C2

        Attachments

          Activity

            People

            • Assignee:
              benedict Benedict Elliott Smith
              Reporter:
              benedict Benedict Elliott Smith
              Authors:
              Benedict Elliott Smith
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: