Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14336

Procedure V2 Phase 1 - Procedure Framework and Making DDL Operations fault tolerant

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Major
    • Resolution: Done
    • 1.1.0, 1.2.0, 1.3.0, 2.0.0
    • None
    • proc-v2

    Description

      This is the first phase of Procedure V2 (HBASE-12439)

      • Core framework
      • re-implement Namespace/Table/Column DDLs to multi-steps procedure with a rollback/rollforward ability in case of failure

      Attachments

        Issue Links

        1.
        Procedure v2 - core framework Sub-task Closed Matteo Bertozzi   Actions
        2.
        Procedure v2 - master create/delete table Sub-task Closed Matteo Bertozzi   Actions
        3.
        Procedure v2 - client create/delete table sync Sub-task Closed Matteo Bertozzi   Actions
        4.
        Procedure V2 - master Add/Modify/Delete Column Family Sub-task Closed Stephen Yuan Jiang

        100%

        Original Estimate - 168h Original Estimate - 168h
        Time Spent - 384h
        Actions
        5.
        Procedure V2 - master Modify table Sub-task Closed Stephen Yuan Jiang

        100%

        Original Estimate - 72h Original Estimate - 72h
        Time Spent - 168h
        Actions
        6.
        Procedure V2 - master Enable/Disable table Sub-task Closed Stephen Yuan Jiang

        100%

        Original Estimate - 120h Original Estimate - 120h
        Time Spent - 216h
        Actions
        7.
        Procedure V2 - master Create/Modify/Delete namespace Sub-task Closed Stephen Yuan Jiang

        100%

        Original Estimate - 168h
        Time Spent - 120h Time Not Required
        Actions
        8.
        Procedure v2 - client enable/disable table sync Sub-task Closed Stephen Yuan Jiang

        100%

        Original Estimate - 120h
        Time Spent - 72h Time Not Required
        Actions
        9.
        Procedure V2 - Use nonces for double submits from client Sub-task Closed Stephen Yuan Jiang   Actions
        10.
        [branch-1.1] Procedure V2 - Make procedure v2 configurable in branch-1.1 Sub-task Closed Stephen Yuan Jiang   Actions
        11.
        Procedure V2 - master truncate table Sub-task Closed Matteo Bertozzi   Actions
        12.
        High level Integration test for master DDL operations Sub-task Closed Sophia Feng   Actions
        13.
        Procedure v2 - WAL Improvements Sub-task Closed Matteo Bertozzi   Actions
        14.
        Procedure V2 - Procedure classes should not be InterfaceAudience.Public Sub-task Closed Enis Soztutar   Actions
        15.
        Procedure v2 - Add maxProcId to the wal header Sub-task Closed Matteo Bertozzi   Actions
        16.
        Improve procedure yielding Sub-task Closed Matteo Bertozzi   Actions
        17.
        Procedure V2: master fail to start due to WALProcedureStore sync failures when HDFS data nodes count is low Sub-task Closed Matteo Bertozzi   Actions
        18.
        Add a NoopProcedureStore for testing Sub-task Closed Matteo Bertozzi   Actions
        19.
        WALProcedureStore fencing is not effective if new WAL rolls Sub-task Closed Enis Soztutar   Actions
        20.
        Procedure v2 - MasterProcedureQueue fix concurrency issue on table queue deletion Sub-task Closed Matteo Bertozzi   Actions
        21.
        Undo workarounds in IntegrationTestDDLMasterFailover for client double submit Sub-task Closed Stephen Yuan Jiang   Actions
        22.
        Procedure V2 - Administrative Task: Provide an API to List all procedures Sub-task Closed Stephen Yuan Jiang   Actions
        23.
        Procedure V2 - Administrative Task: provide an API to abort a procedure Sub-task Closed Stephen Yuan Jiang   Actions
        24.
        Add IT test for procedure-v2-based namespace DDL Sub-task Closed Stephen Yuan Jiang   Actions
        25.
        Move ServerShutdownHandler to Pv2 Sub-task Closed Michael Stack   Actions
        26.
        Procedure V2 - shell command to list all procedures Sub-task Closed Stephen Yuan Jiang   Actions
        27.
        Procedure V2 - shell command to abort a procedure Sub-task Closed Stephen Yuan Jiang   Actions
        28.
        Procedure V2 - enforce ACL on procedure admin tasks Sub-task Closed Stephen Yuan Jiang   Actions
        29.
        Add test to verify DDL operations from admin when the master is restarted Sub-task Closed Unassigned   Actions
        30.
        Procedure v2 - Avoid double yield Sub-task Closed Matteo Bertozzi   Actions
        31.
        Procedure V2 - Remove synchronized around nonce in Procedure submit Sub-task Closed Matteo Bertozzi   Actions
        32.
        Move TestTableDeleteFamilyHandler and TestTableDescriptorModification handler tests to procedure Sub-task Closed Matteo Bertozzi   Actions
        33.
        Split TestWALProcedureStore Sub-task Closed Matteo Bertozzi   Actions
        34.
        Fix ProcedureTestingUtility#LoadCounter.setMaxProcId() Sub-task Closed Matteo Bertozzi   Actions
        35.
        Procedure v2 - fix master crash for FileNotFound Sub-task Closed Matteo Bertozzi   Actions
        36.
        Procedure v2 - use consts for conf properties in tests Sub-task Closed Matteo Bertozzi   Actions
        37.
        Procedure v2 - complete child procedure support Sub-task Closed Matteo Bertozzi   Actions
        38.
        Procedure v2 - Make ProcedureException extend HBaseException Sub-task Closed Matteo Bertozzi   Actions
        39.
        Procedure v2 - Test WAL protobuf entry size limit Sub-task Closed Matteo Bertozzi   Actions
        40.
        Procedure v2 - Cache system user and avoid IOException Sub-task Closed Matteo Bertozzi   Actions
        41.
        Procedure v2 - Extract chore from the executor Sub-task Closed Matteo Bertozzi   Actions
        42.
        Procedure-V2: Enforce procedure ownership at submission Sub-task Closed Ted Yu   Actions
        43.
        Procedure v2 - Cleanup sched wait/lock semantic Sub-task Closed Matteo Bertozzi   Actions
        44.
        Procedure v2 - Cleanup suspended proc execution Sub-task Closed Matteo Bertozzi   Actions
        45.
        Procedure v2 - Fix yield while holding locks Sub-task Closed Matteo Bertozzi   Actions
        46.
        Procedure V2 - Fix nonce submission to avoid unnecessary calling coprocessor multiple times Sub-task Closed Matteo Bertozzi   Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            syuanjiang Stephen Yuan Jiang
            syuanjiang Stephen Yuan Jiang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 648h Original Estimate - 648h
                648h
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 960h
                960h

                Slack

                  Issue deployment