[HBASE-14336] Procedure V2 Phase 1 - Procedure Framework and Making DDL Operations fault tolerant

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Done
• Affects Version/s: 1.1.0, 1.2.0, 1.3.0, 2.0.0
• Component/s: proc-v2
• Labels: reliability
• Assignee: Stephen Yuan Jiang
• Reporter: Stephen Yuan Jiang
• Created: 29/Aug/15 04:10
• Updated: 01/Oct/15 19:41
• Resolved: 01/Oct/15 19:41

Description

This is the first phase of Procedure V2 (HBASE-12439)

• Core framework
• re-implement Namespace/Table/Column DDLs to multi-steps procedure with a rollback/rollforward ability in case of failure

Issue Links

• Is contained by: HBASE-12439 Procedure V2

Sub-Tasks

• Procedure v2 - core framework: Closed
• Procedure v2 - master create/delete table: Closed
• Procedure v2 - client create/delete table sync: Closed
• Procedure V2 - master Add/Modify/Delete Column Family: Closed
• Procedure V2 - master Modify table: Closed
• Procedure V2 - master Enable/Disable table: Closed
• Procedure V2 - master Create/Modify/Delete namespace: Resolved
• Procedure v2 - client enable/disable table sync: Closed
• Procedure V2 - Use nonces for double submits from client: Closed
• [branch-1.1] Procedure V2 - Make procedure v2 configurable in branch-1.1: Closed
• Procedure V2 - master truncate table: Closed
• High level Integration test for master DDL operations: Resolved
• Procedure v2 - WAL Improvements: Closed
• Procedure V2 - Procedure classes should not be InterfaceAudience.Public: Closed
• Procedure v2 - Add maxProcId to the wal header: Closed
• Improve procedure yielding: Resolved
• Procedure V2: master fail to start due to WALProcedureStore sync failures when HDFS data nodes count is low: Resolved
• Add a NoopProcedureStore for testing: Resolved
• WALProcedureStore fencing is not effective if new WAL rolls: Resolved
• Procedure v2 - MasterProcedureQueue fix concurrency issue on table queue deletion: Resolved
• Undo workarounds in IntegrationTestDDLMasterFailover for client double submit: Resolved
• Procedure V2 - Administrative Task: Provide an API to List all procedures: Closed
• Procedure V2 - Administrative Task: provide an API to abort a procedure: Closed
• Add IT test for procedure-v2-based namespace DDL: Resolved
• Move ServerShutdownHandler to Pv2: Resolved
• Procedure V2 - shell command to list all procedures: Closed
• Procedure V2 - shell command to abort a procedure: Closed
• Procedure V2 - enforce ACL on procedure admin tasks: Closed
• Add test to verify DDL operations from admin when the master is restarted: Resolved
• Procedure v2 - Avoid double yield: Resolved
• Procedure V2 - Remove synchronized around nonce in Procedure submit: Resolved
• Move TestTableDeleteFamilyHandler and TestTableDescriptorModification handler tests to procedure: Resolved
• Split TestWALProcedureStore: Resolved
• Fix ProcedureTestingUtility#LoadCounter.setMaxProcId(): Resolved
• Procedure v2 - fix master crash for FileNotFound: Resolved
• Procedure v2 - use consts for conf properties in tests: Resolved
• Procedure v2 - complete child procedure support: Resolved
• Procedure v2 - Make ProcedureException extend HBaseException: Resolved
• Procedure v2 - Test WAL protobuf entry size limit: Resolved
• Procedure v2 - Cache system user and avoid IOException: Resolved
• Procedure v2 - Extract chore from the executor: Resolved
• Procedure-V2: Enforce procedure ownership at submission: Resolved
• Procedure v2 - Cleanup sched wait/lock semantic: Resolved
• Procedure v2 - Cleanup suspended proc execution: Resolved
• Procedure v2 - Fix yield while holding locks: Resolved
• Procedure V2 - Fix nonce submission to avoid unnecessary calling coprocessor multiple times: Closed