Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-871

Mutable Secondary Indexing can cause a deadlock on exception

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.3, 3.0.0, 4.0.0
    • 3.0.0, 4.0.0
    • None
    • None

    Description

      We had a regionserver fail to go down on our internal environment. After digging through the jstacks and logs, it looks like it was because the INDEX_UPDATE_LOCK was held and never released because, we think, and exception writing do the WAL, meaning the postXXXX methods never got called.

      We then realized that we can remove the lock entirely (and the need for the log roll synchronization) by performing the write during postBatchMutate. This avoids an possible hanging locks (from not getting informed of the exception) but still retains the rest of the index correctness since it is performed under the updateLock (which will prevent a cache flush or a log roll).

      Attachments

        1. phoenix-871-3.0.txt
          13 kB
          Jesse Yates
        2. phoenix-871-master.txt
          14 kB
          Jesse Yates

        Activity

          People

            jesse_yates Jesse Yates
            jesse_yates Jesse Yates
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: