Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14350

Procedure V2 Phase 2: Assignment Manager

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0-beta-1, 2.0.0
    • master, proc-v2
    • None
    • Hide
      (Incomplete)

      = Incompatbiles

      == Coprocessor Incompatibilities

      Split/Merge have moved to the Master; it runs them now. Means hooks around Split/Merge are now noops. To intercept Split/Merge phases, CPs need to intercept on MasterObserver.
      Show
      (Incomplete) = Incompatbiles == Coprocessor Incompatibilities Split/Merge have moved to the Master; it runs them now. Means hooks around Split/Merge are now noops. To intercept Split/Merge phases, CPs need to intercept on MasterObserver.

    Description

      This is the second phase of Procedure V2 (HBASE-12439). Built on top of Phase 1 (HBASE-14336), implement a new Assignment Manager based on Proc-V2.

      Attachments

        Issue Links

        1.
        Procedure v2: Core Assignment Manager Sub-task Resolved Matteo Bertozzi Actions
        2.
        Procedure v2: Replace the old AM with the new AM Sub-task Resolved Matteo Bertozzi Actions
        3.
        Procedure V2: Implement move shell command to use Proc-V2 assignment Sub-task Resolved Stephen Yuan Jiang Actions
        4.
        Procedure V2: unnecessaery operations on AssignmentManager#recoverTableInDisablingState() and recoverTableInEnablingState() Sub-task Resolved Stephen Yuan Jiang Actions
        5.
        Procedure V2: Implement single region assignment procedure from Master-side Sub-task Resolved Stephen Yuan Jiang Actions
        6.
        Procedure V2: Update ServerCrashProcedure to interact with assignment procedures Sub-task Resolved Matteo Bertozzi Actions
        7.
        Remove unused LoadBalancer.immediateAssignment() Sub-task Resolved Matteo Bertozzi Actions
        8.
        Procedure V2 - Completed support parent-child procedure Sub-task Resolved Stephen Yuan Jiang Actions
        9.
        Replace master.am.getTableStateManager() with the direct master.getTableStateManager() Sub-task Resolved Matteo Bertozzi Actions
        10.
        replace master.am and master.sm direct access with getter calls Sub-task Resolved Matteo Bertozzi Actions
        11.
        Remove HMaster.assignRegion() Sub-task Resolved Matteo Bertozzi Actions
        12.
        Procedure V2 - Procedure Queue with Regions Sub-task Resolved Matteo Bertozzi Actions
        13.
        Procedure v2 - Improve cleaning up of proc wals Sub-task Resolved Apekshit Sharma Actions
        14.
        Procedure v2 - Reimplement split Sub-task Resolved Stephen Yuan Jiang Actions
        15.
        Procedure v2 - Reimplement merge Sub-task Resolved Stephen Yuan Jiang Actions
        16.
        Require only MasterServices to the ServerManager constructor Sub-task Resolved Matteo Bertozzi Actions
        17.
        Procedure V2: Reimplement DispatchMergingRegionHandler Sub-task Resolved Stephen Yuan Jiang Actions
        18.
        Move meta bootstrap out of HMaster Sub-task Resolved Matteo Bertozzi Actions
        19.
        Procedure V2: Support acquire/release shared table lock concurrently Sub-task Resolved Stephen Yuan Jiang Actions
        20.
        Procedure V2 - Add support to addChildProcedure() as last "step" in StateMachineProcedure Sub-task Resolved Matteo Bertozzi Actions
        21.
        Separate Create/Modify Table operations from open/reopen regions Sub-task Resolved Matteo Bertozzi Actions
        22.
        Procedure v2 - Add handling of split region transition to the new AM Sub-task Resolved Matteo Bertozzi Actions
        23.
        Procedure v2 - Add new AM metrics Sub-task Resolved Umesh Agashe Actions
        24.
        Procedure V2 - Recover 'updated' part of WAL tracker if trailer is corrupted. Sub-task Resolved Apekshit Sharma Actions
        25.
        Procedure v2 - Compute WALs cleanup on wal modification and not on every sync Sub-task Resolved Matteo Bertozzi Actions
        26.
        Procedure v2 - Support for parent holding locks Sub-task Resolved Matteo Bertozzi Actions
        27.
        Procedure V2 - Lock procedures to allow clients to acquire locks on tables/namespaces/regions Sub-task Resolved Matteo Bertozzi Actions
        28.
        Procedure V2 - Move ZK-lock's uses to Procedure framework locks (LockProcedure) Sub-task Resolved Matteo Bertozzi Actions
        29.
        Procedure V2 - Remove org.apache.hadoop.hbase.zookeeper.lock Sub-task Resolved Apekshit Sharma Actions
        30.
        Procedure v2 - Fix StateMachineProcedure support for child procs at last step Sub-task Resolved Matteo Bertozzi Actions
        31.
        Procedure v2 - Inherit lock from root proc Sub-task Resolved Matteo Bertozzi Actions
        32.
        Procedure V2 - Check ACLs for remote HBaseLock Sub-task Resolved Apekshit Sharma Actions
        33.
        Procedure v2 - add waiting procs back to the queue after restart Sub-task Resolved Matteo Bertozzi Actions
        34.
        Procedure v2 - A couple of tweaks to the SplitTableRegionProcedure Sub-task Resolved Matteo Bertozzi Actions
        35.
        Procedure v2 - A couple of tweaks to the SplitTableRegionProcedure Sub-task Resolved Matteo Bertozzi Actions
        36.
        Procedure v2 - remove tryAcquire*Lock and use wait/wake to make framework event based Sub-task Resolved Matteo Bertozzi Actions
        37.
        Procedure v2 - inherit region locks Sub-task Resolved Matteo Bertozzi Actions
        38.
        Remove merge region code from region server Sub-task Resolved Stephen Yuan Jiang Actions
        39.
        Remove split region code from Region Server Sub-task Resolved Stephen Yuan Jiang Actions
        40.
        Procedure v2 - cleanup isSuspended from MasterProcedureScheduler#Queue Sub-task Resolved Apekshit Sharma Actions
        41.
        Procedure v2 - Add handling of merge region transition to the new AM Sub-task Resolved Matteo Bertozzi Actions
        42.
        Readd TestMergeTableRegionsProcedure mistakenly removed by HBASE-16786 Sub-task Resolved Michael Stack Actions
        43.
        Tooling to move table out of FAILED_OPEN state Sub-task Resolved Unassigned Actions
        44.
        Create LoadBalancer perf-tests (test balancer algorithm decoupled from workload) Sub-task Resolved Umesh Agashe Actions
        45.
        Subset of HBASE-14614, Procedure v2: Core Assignment Manager (non-critical related changes) Sub-task Resolved Michael Stack Actions
        46.
        Implement abort for TruncateTableProcedure Sub-task Resolved Umesh Agashe Actions
        47.
        Support abort for all procedures by default Sub-task Resolved Umesh Agashe Actions
        48.
        Fix unit tests for TestTableFavoredNodes Sub-task Resolved Umesh Agashe Actions
        49.
        Add API for who currently holds a lock on namespace/ table/ region and log when state is LOCK_EVENT_WAIT Sub-task Resolved Umesh Agashe Actions
        50.
        [SHELL] Purge close_region command that allows by-pass of Master Sub-task Resolved Apekshit Sharma Actions
        51.
        [AMv2] If Master gives OPEN to another, if original eventually succeeds, Master will kill it Sub-task Resolved Umesh Agashe Actions
        52.
        [AMv2] Enable aggregation of RPCs (assigns/unassigns, etc.) Sub-task Resolved Umesh Agashe Actions
        53.
        [AMv2] Split/Merge need cleanup; currently they diverge and do not fully embrace AMv2 world Sub-task Resolved Yi Liang Actions
        54.
        Redo ProcedureInfo and LockInfo Sub-task Resolved Balazs Meszaros Actions
        55.
        [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions Sub-task Resolved Yi Liang Actions
        56.
        Procedure WALs are archived but not cleaned; fix Sub-task Resolved Peter Somogyi Actions
        57.
        Assign system tables first (priority) Sub-task Resolved Yi Liang Actions
        58.
        Verify Master carrying system-tables only with short-circuit RPC Sub-task Resolved Michael Stack Actions
        59.
        create new Async Split API to embrace AM v2 Sub-task Resolved Yi Liang Actions
        60.
        Deprecate and throw unsupported operation when Admin#closeRegion is called. Sub-task Resolved Apekshit Sharma Actions
        61.
        Reduce ProcedureInfo usage Sub-task Resolved Balazs Meszaros Actions
        62.
        Add pv2 and amv2 chapters to refguide Sub-task Resolved Michael Stack Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            syuanjiang Stephen Yuan Jiang
            syuanjiang Stephen Yuan Jiang
            Votes:
            0 Vote for this issue
            Watchers:
            23 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment