Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-11165 Scaling so cluster can host 1M regions and beyond (50M regions?)
  3. HBASE-11059

ZK-less region assignment

    XMLWordPrintableJSON

Details

    • Reviewed
    • Hide
      This patch makes it possible to do region assignment without ZK. By default, it is off (i.e. ZK is used for region assignment as before).

      Two setting "hbase.assignment.usezk", "hbase.assignment.usezk.migrating" are introduced to control migration from using ZK for assignment to not using
      ZK for assignment.

      For rolling upgrade from using ZK to not using ZK, it can be done in two steps:

      1. Set both hbase.assignment.usezk and hbase.assignment.usezk.migrating to true, do a rolling upgrade so that both masters and regionservers have
      the new code. Either master or regionserver can be upgraded at first. The order is not important for this step. If you want to keep using ZK for assignment, you'd better set hbase.assignment.usezk to true, and hbase.assignment.usezk.migrating to false so that region states are not persisted in meta table.

      2. Set hbase.assignment.usezk to false, do a rolling restart so that region assignments don't use ZK any more. For this step, masters should be restarted after regionservers have all restarted at first so that they won't update meta table directly and master doesn't know about it.
      Show
      This patch makes it possible to do region assignment without ZK. By default, it is off (i.e. ZK is used for region assignment as before). Two setting "hbase.assignment.usezk", "hbase.assignment.usezk.migrating" are introduced to control migration from using ZK for assignment to not using ZK for assignment. For rolling upgrade from using ZK to not using ZK, it can be done in two steps: 1. Set both hbase.assignment.usezk and hbase.assignment.usezk.migrating to true, do a rolling upgrade so that both masters and regionservers have the new code. Either master or regionserver can be upgraded at first. The order is not important for this step. If you want to keep using ZK for assignment, you'd better set hbase.assignment.usezk to true, and hbase.assignment.usezk.migrating to false so that region states are not persisted in meta table. 2. Set hbase.assignment.usezk to false, do a rolling restart so that region assignments don't use ZK any more. For this step, masters should be restarted after regionservers have all restarted at first so that they won't update meta table directly and master doesn't know about it.

    Description

      It seems that most people don't like region assignment with ZK (HBASE-5487), which causes many uncertainties. This jira is to support ZK-less region assignment. We need to make sure this patch doesn't break backward compatibility/rolling upgrade.

      Attachments

        1. hbase-11059_v2.1.patch
          286 kB
          Jimmy Xiang
        2. hbase-11059_v2.2.patch
          296 kB
          Jimmy Xiang
        3. hbase-11059_v2.patch
          261 kB
          Jimmy Xiang
        4. hbase-11059_v3.0.patch
          309 kB
          Jimmy Xiang
        5. hbase-11059.patch
          289 kB
          Jimmy Xiang
        6. zk-less_am.pdf
          119 kB
          Jimmy Xiang
        7. zk-less_assignment.png
          20 kB
          Jimmy Xiang

        Issue Links

          is related to

          Umbrella - An overarching type made of sub-tasks HBASE-19997 [rolling upgrade] 1.x => 2.x

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          relates to

          Sub-task - The sub-task of the issue HBASE-11546 Backport ZK-less region assignment to 0.98

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-18619 Should we add a postOpenDeployTasks after open splited or merged region?

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HBASE-11659 Region state RPC call is not idempotent

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-11793 RegionStates shouldn't be locked while writing to META

          • Major - Major loss of function.
          • Closed

          Umbrella - An overarching type made of sub-tasks HBASE-10909 Abstract out ZooKeeper usage in HBase - phase 1

          • Major - Major loss of function.
          • Closed

          Activity

            People

              jxiang Jimmy Xiang
              jxiang Jimmy Xiang
              Votes:
              0 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: