Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14621

ReplicationLogCleaner gets stuck when a regionserver crashes

    Details

    • Hadoop Flags:
      Reviewed

      Description

      The ReplicationLogCleaner has a bug that makes it get stuck in an infinite loop when a regionserver crashes. This bug was introduced in the fix for HBASE-12865; which makes sure that the loadWALsFromQueues method attempts a retry whenever the replication node's cversion is changed in the middle of loading the replication queue for the regionservers. However, if this scenario actually happens (a regionserver crash in the middle of the operation), it will get stuck in an infinite loop.

      It has very serious ramifications because the old WALs are not cleaned up because of this and in a high load environment, the file count in the oldWALs directory soon exceeds the inode limit and the cluster goes down.

        Attachments

        1. HBASE-14621-1.patch
          4 kB
          Ashu Pachauri
        2. HBASE-14621.patch
          1 kB
          Ashu Pachauri

          Activity

            People

            • Assignee:
              THEcreationist Ashu Pachauri
              Reporter:
              THEcreationist Ashu Pachauri
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: