[HBASE-21375] Revisit the lock and queue implementation in MasterProcedureScheduler

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.2.0, 2.0.3, 2.1.2
• Component/s: proc-v2
• Hadoop Flags: Reviewed
• Assignee: Duo Zhang
• Reporter: Duo Zhang
• Created: 24/Oct/18 01:50
• Updated: 23/Apr/19 11:51
• Resolved: 29/Oct/18 12:45

Description

The problem for the old implementation is that we will only check the first procedure in a queue to see if it could run, if it can not, we will remove the queue from run queue. So when adding procedure to the scheduler, we have to try hard to put the procedure which can be executed in front of the queue, if there are corner cases where we fail to do so, it will likely lead to a dead lock, that's why we have the tricky code when loading procedures and try to add them into the scheduler, and also lots of 'if' in the doAdd method of MasterProcedureScheduler. But this is still not enough to make things right, so finally allan163 and I decided to change the logic in doPoll method, where we use a loop to find whether there is a procedure can be executed, not only the first one.

Issue Links

• is related to: HBASE-21364 Procedure holds the lock should put to front of the queue after restart
• links to: Review Board