Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20828 Finish-up AMv2 Design/List of Tenets/Specification of operation
  3. HBASE-21375

Revisit the lock and queue implementation in MasterProcedureScheduler

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha-1, 2.2.0, 2.0.3, 2.1.2
    • Component/s: proc-v2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The problem for the old implementation is that we will only check the first procedure in a queue to see if it could run, if it can not, we will remove the queue from run queue. So when adding procedure to the scheduler, we have to try hard to put the procedure which can be executed in front of the queue, if there are corner cases where we fail to do so, it will likely lead to a dead lock, that's why we have the tricky code when loading procedures and try to add them into the scheduler, and also lots of 'if' in the doAdd method of MasterProcedureScheduler. But this is still not enough to make things right, so finally Allan Yang and I decided to change the logic in doPoll method, where we use a loop to find whether there is a procedure can be executed, not only the first one.

        Attachments

        1. HBASE-21375.patch
          34 kB
          Duo Zhang
        2. HBASE-21375-UT.patch
          9 kB
          Duo Zhang
        3. HBASE-21375-UT2.patch
          16 kB
          Duo Zhang
        4. HBASE-21375-v1.patch
          74 kB
          Duo Zhang
        5. HBASE-21375-v2.patch
          74 kB
          Duo Zhang

          Issue Links

            Activity

              People

              • Assignee:
                zhangduo Duo Zhang
                Reporter:
                zhangduo Duo Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: