Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14336

Procedure V2 Phase 1 - Procedure Framework and Making DDL Operations fault tolerant

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 1.1.0, 1.2.0, 1.3.0, 2.0.0
    • Fix Version/s: None
    • Component/s: proc-v2
    • Labels:

      Description

      This is the first phase of Procedure V2 (HBASE-12439)

      • Core framework
      • re-implement Namespace/Table/Column DDLs to multi-steps procedure with a rollback/rollforward ability in case of failure

        Attachments

          Issue Links

          1.
          Procedure v2 - core framework Sub-task Closed Matteo Bertozzi  
          2.
          Procedure v2 - master create/delete table Sub-task Closed Matteo Bertozzi  
          3.
          Procedure v2 - client create/delete table sync Sub-task Closed Matteo Bertozzi  
          4.
          Procedure V2 - master Add/Modify/Delete Column Family Sub-task Closed Stephen Yuan Jiang

          100%

          Original Estimate - 168h Original Estimate - 168h
          Time Spent - 384h
          5.
          Procedure V2 - master Modify table Sub-task Closed Stephen Yuan Jiang

          100%

          Original Estimate - 72h Original Estimate - 72h
          Time Spent - 168h
          6.
          Procedure V2 - master Enable/Disable table Sub-task Closed Stephen Yuan Jiang

          100%

          Original Estimate - 120h Original Estimate - 120h
          Time Spent - 216h
          7.
          Procedure V2 - master Create/Modify/Delete namespace Sub-task Resolved Stephen Yuan Jiang

          100%

          Original Estimate - 168h
          Time Spent - 120h Time Not Required
          8.
          Procedure v2 - client enable/disable table sync Sub-task Closed Stephen Yuan Jiang

          100%

          Original Estimate - 120h
          Time Spent - 72h Time Not Required
          9.
          Procedure V2 - Use nonces for double submits from client Sub-task Closed Stephen Yuan Jiang  
          10.
          [branch-1.1] Procedure V2 - Make procedure v2 configurable in branch-1.1 Sub-task Closed Stephen Yuan Jiang  
          11.
          Procedure V2 - master truncate table Sub-task Closed Matteo Bertozzi  
          12.
          High level Integration test for master DDL operations Sub-task Resolved Sophia Feng  
          13.
          Procedure v2 - WAL Improvements Sub-task Closed Matteo Bertozzi  
          14.
          Procedure V2 - Procedure classes should not be InterfaceAudience.Public Sub-task Closed Enis Soztutar  
          15.
          Procedure v2 - Add maxProcId to the wal header Sub-task Closed Matteo Bertozzi  
          16.
          Improve procedure yielding Sub-task Resolved Matteo Bertozzi  
          17.
          Procedure V2: master fail to start due to WALProcedureStore sync failures when HDFS data nodes count is low Sub-task Resolved Matteo Bertozzi  
          18.
          Add a NoopProcedureStore for testing Sub-task Resolved Matteo Bertozzi  
          19.
          WALProcedureStore fencing is not effective if new WAL rolls Sub-task Resolved Enis Soztutar  
          20.
          Procedure v2 - MasterProcedureQueue fix concurrency issue on table queue deletion Sub-task Resolved Matteo Bertozzi  
          21.
          Undo workarounds in IntegrationTestDDLMasterFailover for client double submit Sub-task Resolved Stephen Yuan Jiang  
          22.
          Procedure V2 - Administrative Task: Provide an API to List all procedures Sub-task Closed Stephen Yuan Jiang  
          23.
          Procedure V2 - Administrative Task: provide an API to abort a procedure Sub-task Closed Stephen Yuan Jiang  
          24.
          Add IT test for procedure-v2-based namespace DDL Sub-task Resolved Stephen Yuan Jiang  
          25.
          Move ServerShutdownHandler to Pv2 Sub-task Resolved stack  
          26.
          Procedure V2 - shell command to list all procedures Sub-task Closed Stephen Yuan Jiang  
          27.
          Procedure V2 - shell command to abort a procedure Sub-task Closed Stephen Yuan Jiang  
          28.
          Procedure V2 - enforce ACL on procedure admin tasks Sub-task Closed Stephen Yuan Jiang  
          29.
          Add test to verify DDL operations from admin when the master is restarted Sub-task Resolved Matteo Bertozzi  
          30.
          Procedure v2 - Avoid double yield Sub-task Resolved Matteo Bertozzi  
          31.
          Procedure V2 - Remove synchronized around nonce in Procedure submit Sub-task Resolved Matteo Bertozzi  
          32.
          Move TestTableDeleteFamilyHandler and TestTableDescriptorModification handler tests to procedure Sub-task Resolved Matteo Bertozzi  
          33.
          Split TestWALProcedureStore Sub-task Resolved Matteo Bertozzi  
          34.
          Fix ProcedureTestingUtility#LoadCounter.setMaxProcId() Sub-task Resolved Matteo Bertozzi  
          35.
          Procedure v2 - fix master crash for FileNotFound Sub-task Resolved Matteo Bertozzi  
          36.
          Procedure v2 - use consts for conf properties in tests Sub-task Resolved Matteo Bertozzi  
          37.
          Procedure v2 - complete child procedure support Sub-task Resolved Matteo Bertozzi  
          38.
          Procedure v2 - Make ProcedureException extend HBaseException Sub-task Resolved Matteo Bertozzi  
          39.
          Procedure v2 - Test WAL protobuf entry size limit Sub-task Resolved Matteo Bertozzi  
          40.
          Procedure v2 - Cache system user and avoid IOException Sub-task Resolved Matteo Bertozzi  
          41.
          Procedure v2 - Extract chore from the executor Sub-task Resolved Matteo Bertozzi  
          42.
          Procedure-V2: Enforce procedure ownership at submission Sub-task Resolved Ted Yu  
          43.
          Procedure v2 - Cleanup sched wait/lock semantic Sub-task Resolved Matteo Bertozzi  
          44.
          Procedure v2 - Cleanup suspended proc execution Sub-task Resolved Matteo Bertozzi  
          45.
          Procedure v2 - Fix yield while holding locks Sub-task Resolved Matteo Bertozzi  
          46.
          Procedure V2 - Fix nonce submission to avoid unnecessary calling coprocessor multiple times Sub-task Closed Matteo Bertozzi  

            Activity

              People

              • Assignee:
                syuanjiang Stephen Yuan Jiang
                Reporter:
                syuanjiang Stephen Yuan Jiang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 648h Original Estimate - 648h
                  648h
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 960h
                  960h