Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15800

BinLog deadlock on stopping when the sample queue is full

    XMLWordPrintableJSON

    Details

    • Bug Category:
      Degradation - Resource Management
    • Severity:
      Normal
    • Complexity:
      Normal
    • Discovered By:
      Code Inspection
    • Platform:
      All
    • Impacts:
      None

      Description

      A deadlock can happen when 1) the BinLog is being stoped and 2) the BinLog's internal sample queue is full.

      When stopping, BinLog first set the flag shouldContinue to false, so that the internal consumer thread stop consuming. It is possible to leave the queue being full.
      Then, the BinLog puts one extra object NO_OP into the sample queue. However, the queue is already full, so the put operation blocks, and the stop method never returns.
      Therefore, we got a deadlock.

      BinLog is used by Cassandra 40 new features such as audit logging and full query logging.
      If such deadlock happens, the thread cannot be not joined and the referenced items in the queue are never released, hence memory leak.

        Attachments

          Activity

            People

            • Assignee:
              yifanc Yifan Cai
              Reporter:
              yifanc Yifan Cai
              Authors:
              Yifan Cai
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: