Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-12770

Don't transfer all the queued hlogs of a dead server to the same alive server

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.0, 2.0.0
    • Fix Version/s: 1.4.0, 1.3.1, 2.0.0
    • Component/s: Replication
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When a region server is down(or the cluster restart), all the hlog queues will be transferred by the same alive region server. In a shared cluster, we might create several peers replicating data to different peer clusters. There might be lots of hlogs queued for these peers caused by several reasons, such as some peers might be disabled, or errors from peer cluster might prevent the replication, or the replication sources may fail to read some hlog because of hdfs problem. Then, if the server is down or restarted, another alive server will take all the replication jobs of the dead server, this might bring a big pressure to resources(network/disk read) of the alive server and also is not fast enough to replicate the queued hlogs. And if the alive server is down, all the replication jobs including that takes from other dead servers will once again be totally transferred to another alive server, this might cause a server have a large number of queued hlogs(in our shared cluster, we find one server might have thousands of queued hlogs for replication). As an optional way, is it reasonable that the alive server only transfer one peer's hlogs from the dead server one time? Then, other alive region servers might have the opportunity to transfer the hlogs of rest peers. This may also help the queued hlogs be processed more fast. Any discussion is welcome.

        Attachments

        1. HBASE-12770-trunk.patch
          20 kB
          Jianwei Cui
        2. HBASE-12770-v1.patch
          37 kB
          Phil Yang
        3. HBASE-12770-branch-1-v1.patch
          27 kB
          Phil Yang
        4. HBASE-12770-v2.patch
          37 kB
          Phil Yang
        5. HBASE-12770-branch-1-v2.patch
          27 kB
          Phil Yang
        6. HBASE-12770-v3.patch
          37 kB
          Phil Yang
        7. HBASE-12770-branch-1-v3.patch
          28 kB
          Phil Yang
        8. HBASE-12770-branch-1-v3.patch
          28 kB
          Phil Yang
        9. HBASE-12770-v3.patch
          37 kB
          Phil Yang
        10. HBASE-12770-branch-1-v3.patch
          28 kB
          Phil Yang
        11. HBASE-12770-branch-1-v3.patch
          28 kB
          Phil Yang

          Issue Links

            Activity

              People

              • Assignee:
                yangzhe1991 Phil Yang
                Reporter:
                cuijianwei Jianwei Cui
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: