Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-19290

Reduce zk request when doing split log

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha-1, 1.5.0, 2.0.0
    • 1.5.0, 2.0.0
    • None
    • None

    Description

      We observe once the cluster has 1000+ nodes and when hundreds of nodes abort and doing split log, the split is very very slow, and we find the regionserver and master wait on the zookeeper response, so we need to reduce zookeeper request and pressure for big cluster.
      (1) Reduce request to rsZNode, every time calculateAvailableSplitters will get rsZNode's children from zookeeper, when cluster is huge, this is heavy. This patch reduce the request.
      (2) When the regionserver has max split tasks running, it may still trying to grab task and issue zookeeper request, we should sleep and wait until we can grab tasks again.

      Attachments

        1. HBASE-19290.branch-1.001.patch
          8 kB
          Lijin Bin
        2. HBASE-19290.branch-1.001.patch
          8 kB
          Lijin Bin
        3. HBASE-19290.master.001.patch
          7 kB
          Lijin Bin
        4. HBASE-19290.master.002.patch
          7 kB
          Lijin Bin
        5. HBASE-19290.master.003.patch
          7 kB
          Lijin Bin
        6. HBASE-19290.master.004.patch
          8 kB
          Lijin Bin
        7. HBASE-19290.master.005.patch
          8 kB
          Lijin Bin
        8. HBASE-19290.master.006.patch
          8 kB
          Lijin Bin
        9. HBASE-19290.master.006.patch
          8 kB
          Lijin Bin

        Activity

          People

            binlijin Lijin Bin
            binlijin Lijin Bin
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: