Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10401

[hbck] perform overlap group merges in parallel

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1
    • Fix Version/s: 0.98.0, 0.96.2, 0.99.0, 0.94.17
    • Component/s: hbck
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In a recent support case, we encountered a corrupt hbase that had thousands of "overlap groups" (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete.

      This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations.

        Attachments

        1. hbase-10401.v2.patch
          10 kB
          Jonathan Hsieh
        2. hbase-10401.v2.94.patch
          10 kB
          Jonathan Hsieh
        3. hbase-10401.patch
          10 kB
          Jonathan Hsieh
        4. hbase-10401.10400.patch
          10 kB
          Jonathan Hsieh

          Activity

            People

            • Assignee:
              jmhsieh Jonathan Hsieh
              Reporter:
              jmhsieh Jonathan Hsieh
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: