[HBASE-20828] Finish-up AMv2 Design/List of Tenets/Specification of operation

• Type: Umbrella
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: amv2
• Assignee: Unassigned
• Reporter: Michael Stack
• Created: 01/Jul/18 01:54
• Updated: 24/Oct/18 00:39

Description

AMv2 is missing specification. There are too many grey-areas still. Also missing are a concise listing of the tenets of AMv2 operation. Here are some examples:

• HBASE-19529 "Handle null states in AM": Asks how we should treat null state in hbase:meta. What does it 'mean'. We seem to treat it differently dependent on context. Needs clarification. [~Apache9] recently asked similar about the meaning of OFFLINE.
• Logging needs to have a particular form to help trace Procedure progress; needs a write-up.

Lets fill in items to address in this umbrella issue. Can address in subissues and produce specification doc too. We have the below but these are mostly (incomplete) description for devs on pv2 and amv2; the specification is missing:

http://hbase.apache.org/book.html#pv2
http://hbase.apache.org/book.html#amv2

(Other areas include addressing what is up w/ rollback – when, how much, and when it is not appropriate – as well as recommendation on Procedures coarseness, locking – is it ok to lock table in alter table procedure for the life of the procedure? – and so on).

Sub-Tasks

• Handle null states in AM: Open
• Merge the update procedure store on locking with the general persist after a procedure execution: Open
• Procedure worker should not quit when getting unexpected error: Patch Available
• Revisit the close region related code at RS side: Open
• Load procedure wals with multiple threads: Open
• Add more tests for procedure store related classes: Open
• TransitRegionStateProcedure should not fail with FAILED_OPEN when acting as a sub procedure: Open
• Polish the rollback implementation in ProcedureExecutor: Open
• Add more comments about how we do fencing for WALProcedureStore: Open