Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-11165 Scaling so cluster can host 1M regions and beyond (50M regions?)
  3. HBASE-11059

ZK-less region assignment

          Details

          • Hadoop Flags:
            Reviewed
          • Release Note:
            Hide
            This patch makes it possible to do region assignment without ZK. By default, it is off (i.e. ZK is used for region assignment as before).

            Two setting "hbase.assignment.usezk", "hbase.assignment.usezk.migrating" are introduced to control migration from using ZK for assignment to not using
            ZK for assignment.

            For rolling upgrade from using ZK to not using ZK, it can be done in two steps:

            1. Set both hbase.assignment.usezk and hbase.assignment.usezk.migrating to true, do a rolling upgrade so that both masters and regionservers have
            the new code. Either master or regionserver can be upgraded at first. The order is not important for this step. If you want to keep using ZK for assignment, you'd better set hbase.assignment.usezk to true, and hbase.assignment.usezk.migrating to false so that region states are not persisted in meta table.

            2. Set hbase.assignment.usezk to false, do a rolling restart so that region assignments don't use ZK any more. For this step, masters should be restarted after regionservers have all restarted at first so that they won't update meta table directly and master doesn't know about it.
            Show
            This patch makes it possible to do region assignment without ZK. By default, it is off (i.e. ZK is used for region assignment as before). Two setting "hbase.assignment.usezk", "hbase.assignment.usezk.migrating" are introduced to control migration from using ZK for assignment to not using ZK for assignment. For rolling upgrade from using ZK to not using ZK, it can be done in two steps: 1. Set both hbase.assignment.usezk and hbase.assignment.usezk.migrating to true, do a rolling upgrade so that both masters and regionservers have the new code. Either master or regionserver can be upgraded at first. The order is not important for this step. If you want to keep using ZK for assignment, you'd better set hbase.assignment.usezk to true, and hbase.assignment.usezk.migrating to false so that region states are not persisted in meta table. 2. Set hbase.assignment.usezk to false, do a rolling restart so that region assignments don't use ZK any more. For this step, masters should be restarted after regionservers have all restarted at first so that they won't update meta table directly and master doesn't know about it.

            Description

            It seems that most people don't like region assignment with ZK (HBASE-5487), which causes many uncertainties. This jira is to support ZK-less region assignment. We need to make sure this patch doesn't break backward compatibility/rolling upgrade.

              Attachments

              1. hbase-11059_v2.1.patch
                286 kB
                Jimmy Xiang
              2. hbase-11059_v2.2.patch
                296 kB
                Jimmy Xiang
              3. hbase-11059_v2.patch
                261 kB
                Jimmy Xiang
              4. hbase-11059_v3.0.patch
                309 kB
                Jimmy Xiang
              5. hbase-11059.patch
                289 kB
                Jimmy Xiang
              6. zk-less_am.pdf
                119 kB
                Jimmy Xiang
              7. zk-less_assignment.png
                20 kB
                Jimmy Xiang

                Issue Links

                relates to

                Sub-task - The sub-task of the issue HBASE-11546 Backport ZK-less region assignment to 0.98

                • Major - Major loss of function.
                • Closed

                Bug - A problem which impairs or prevents the functions of the product. HBASE-11793 RegionStates shouldn't be locked while writing to META

                • Major - Major loss of function.
                • Open

                Bug - A problem which impairs or prevents the functions of the product. HBASE-18619 Should we add a postOpenDeployTasks after open splited or merged region?

                • Major - Major loss of function.
                • Open

                Bug - A problem which impairs or prevents the functions of the product. HBASE-11659 Region state RPC call is not idempotent

                • Major - Major loss of function.
                • Closed

                Umbrella - An overarching type made of sub-tasks HBASE-10909 Abstract out ZooKeeper usage in HBase - phase 1

                • Major - Major loss of function.
                • Closed

                  Activity

                    People

                    • Assignee:
                      jxiang Jimmy Xiang
                      Reporter:
                      jxiang Jimmy Xiang
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      24 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: