Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14350

Procedure V2 Phase 2: Assignment Manager

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.0.0-beta-1, 2.0.0
    • Component/s: master, proc-v2
    • Labels:
      None
    • Release Note:
      Hide
      (Incomplete)

      = Incompatbiles

      == Coprocessor Incompatibilities

      Split/Merge have moved to the Master; it runs them now. Means hooks around Split/Merge are now noops. To intercept Split/Merge phases, CPs need to intercept on MasterObserver.
      Show
      (Incomplete) = Incompatbiles == Coprocessor Incompatibilities Split/Merge have moved to the Master; it runs them now. Means hooks around Split/Merge are now noops. To intercept Split/Merge phases, CPs need to intercept on MasterObserver.

      Description

      This is the second phase of Procedure V2 (HBASE-12439). Built on top of Phase 1 (HBASE-14336), implement a new Assignment Manager based on Proc-V2.

        Attachments

          Issue Links

          1.
          Procedure v2: Core Assignment Manager Sub-task Resolved Matteo Bertozzi
          2.
          Procedure v2: Replace the old AM with the new AM Sub-task Resolved Matteo Bertozzi
          3.
          Procedure V2: Implement move shell command to use Proc-V2 assignment Sub-task Resolved Stephen Yuan Jiang
          4.
          Procedure V2: unnecessaery operations on AssignmentManager#recoverTableInDisablingState() and recoverTableInEnablingState() Sub-task Resolved Stephen Yuan Jiang
          5.
          Procedure V2: Implement single region assignment procedure from Master-side Sub-task Resolved Stephen Yuan Jiang
          6.
          Procedure V2: Update ServerCrashProcedure to interact with assignment procedures Sub-task Resolved Matteo Bertozzi
          7.
          Remove unused LoadBalancer.immediateAssignment() Sub-task Resolved Matteo Bertozzi
          8.
          Procedure V2 - Completed support parent-child procedure Sub-task Resolved Stephen Yuan Jiang
          9.
          Replace master.am.getTableStateManager() with the direct master.getTableStateManager() Sub-task Resolved Matteo Bertozzi
          10.
          replace master.am and master.sm direct access with getter calls Sub-task Resolved Matteo Bertozzi
          11.
          Remove HMaster.assignRegion() Sub-task Resolved Matteo Bertozzi
          12.
          Procedure V2 - Procedure Queue with Regions Sub-task Resolved Matteo Bertozzi
          13.
          Procedure v2 - Improve cleaning up of proc wals Sub-task Resolved Apekshit Sharma
          14.
          Procedure v2 - Reimplement split Sub-task Resolved Stephen Yuan Jiang
          15.
          Procedure v2 - Reimplement merge Sub-task Resolved Stephen Yuan Jiang
          16.
          Require only MasterServices to the ServerManager constructor Sub-task Resolved Matteo Bertozzi
          17.
          Procedure V2: Reimplement DispatchMergingRegionHandler Sub-task Resolved Stephen Yuan Jiang
          18.
          Move meta bootstrap out of HMaster Sub-task Resolved Matteo Bertozzi
          19.
          Procedure V2: Support acquire/release shared table lock concurrently Sub-task Resolved Stephen Yuan Jiang
          20.
          Procedure V2 - Add support to addChildProcedure() as last "step" in StateMachineProcedure Sub-task Resolved Matteo Bertozzi
          21.
          Separate Create/Modify Table operations from open/reopen regions Sub-task Resolved Matteo Bertozzi
          22.
          Procedure v2 - Add handling of split region transition to the new AM Sub-task Resolved Matteo Bertozzi
          23.
          Procedure v2 - Add new AM metrics Sub-task Resolved Umesh Agashe
          24.
          Procedure V2 - Recover 'updated' part of WAL tracker if trailer is corrupted. Sub-task Resolved Apekshit Sharma
          25.
          Procedure v2 - Compute WALs cleanup on wal modification and not on every sync Sub-task Resolved Matteo Bertozzi
          26.
          Procedure v2 - Support for parent holding locks Sub-task Resolved Matteo Bertozzi
          27.
          Procedure V2 - Lock procedures to allow clients to acquire locks on tables/namespaces/regions Sub-task Resolved Matteo Bertozzi
          28.
          Procedure V2 - Move ZK-lock's uses to Procedure framework locks (LockProcedure) Sub-task Resolved Matteo Bertozzi
          29.
          Procedure V2 - Remove org.apache.hadoop.hbase.zookeeper.lock Sub-task Resolved Apekshit Sharma
          30.
          Procedure v2 - Fix StateMachineProcedure support for child procs at last step Sub-task Resolved Matteo Bertozzi
          31.
          Procedure v2 - Inherit lock from root proc Sub-task Resolved Matteo Bertozzi
          32.
          Procedure V2 - Check ACLs for remote HBaseLock Sub-task Resolved Apekshit Sharma
          33.
          Procedure v2 - add waiting procs back to the queue after restart Sub-task Resolved Matteo Bertozzi
          34.
          Procedure v2 - A couple of tweaks to the SplitTableRegionProcedure Sub-task Resolved Matteo Bertozzi
          35.
          Procedure v2 - A couple of tweaks to the SplitTableRegionProcedure Sub-task Resolved Matteo Bertozzi
          36.
          Procedure v2 - remove tryAcquire*Lock and use wait/wake to make framework event based Sub-task Resolved Matteo Bertozzi
          37.
          Procedure v2 - inherit region locks Sub-task Resolved Matteo Bertozzi
          38.
          Remove merge region code from region server Sub-task Resolved Stephen Yuan Jiang
          39.
          Remove split region code from Region Server Sub-task Resolved Stephen Yuan Jiang
          40.
          Procedure v2 - cleanup isSuspended from MasterProcedureScheduler#Queue Sub-task Resolved Apekshit Sharma
          41.
          Procedure v2 - Add handling of merge region transition to the new AM Sub-task Resolved Matteo Bertozzi
          42.
          Readd TestMergeTableRegionsProcedure mistakenly removed by HBASE-16786 Sub-task Resolved Michael Stack
          43.
          Tooling to move table out of FAILED_OPEN state Sub-task Resolved Unassigned
          44.
          Create LoadBalancer perf-tests (test balancer algorithm decoupled from workload) Sub-task Resolved Umesh Agashe
          45.
          Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes) Sub-task Resolved Michael Stack
          46.
          Implement abort for TruncateTableProcedure Sub-task Resolved Umesh Agashe
          47.
          Support abort for all procedures by default Sub-task Resolved Umesh Agashe
          48.
          Fix unit tests for TestTableFavoredNodes Sub-task Resolved Umesh Agashe
          49.
          Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT Sub-task Resolved Umesh Agashe
          50.
          [SHELL] Purge close_region command that allows by-pass of Master Sub-task Resolved Apekshit Sharma
          51.
          [AMv2] If Master gives OPEN to another, if original eventually succeeds, Master will kill it Sub-task Resolved Umesh Agashe
          52.
          [AMv2] Enable aggregation of RPCs (assigns/unassigns, etc.) Sub-task Resolved Umesh Agashe
          53.
          [AMv2] Split/Merge need cleanup; currently they diverge and do not fully embrace AMv2 world Sub-task Resolved Yi Liang
          54.
          Redo ProcedureInfo and LockInfo Sub-task Resolved Balazs Meszaros
          55.
          [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions Sub-task Resolved Yi Liang
          56.
          Procedure WALs are archived but not cleaned; fix Sub-task Resolved Peter Somogyi
          57.
          Assign system tables first (priority) Sub-task Resolved Yi Liang
          58.
          Verify Master carrying system-tables only with short-circuit RPC Sub-task Resolved Michael Stack
          59.
          create new Async Split API to embrace AM v2 Sub-task Resolved Yi Liang
          60.
          Deprecate and throw unsupported operation when Admin#closeRegion is called. Sub-task Resolved Apekshit Sharma
          61.
          Reduce ProcedureInfo usage Sub-task Resolved Balazs Meszaros
          62.
          Add pv2 and amv2 chapters to refguide Sub-task Resolved Michael Stack

            Activity

              People

              • Assignee:
                syuanjiang Stephen Yuan Jiang
                Reporter:
                syuanjiang Stephen Yuan Jiang
              • Votes:
                0 Vote for this issue
                Watchers:
                21 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: