Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10401

[hbck] perform overlap group merges in parallel

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1
    • 0.98.0, 0.96.2, 0.99.0, 0.94.17
    • hbck
    • None
    • Reviewed

    Description

      In a recent support case, we encountered a corrupt hbase that had thousands of "overlap groups" (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete.

      This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations.

      Attachments

        1. hbase-10401.v2.patch
          10 kB
          Jonathan Hsieh
        2. hbase-10401.v2.94.patch
          10 kB
          Jonathan Hsieh
        3. hbase-10401.patch
          10 kB
          Jonathan Hsieh
        4. hbase-10401.10400.patch
          10 kB
          Jonathan Hsieh

        Activity

          People

            jmhsieh Jonathan Hsieh
            jmhsieh Jonathan Hsieh
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: