Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5916

RS restart just before master intialization we make the cluster non operative

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.92.1, 0.94.0
    • Fix Version/s: 0.92.2, 0.94.1, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Consider a case where my master is getting restarted. RS that was alive when the master restart started, gets restarted before the master initializes the ServerShutDownHandler.

      serverShutdownHandlerEnabled = true;
      

      In this case when the RS tries to register with the master, the master will try to expire the server but the server cannot be expired as still the serverShutdownHandler is not enabled.

      This case may happen when i have only one RS gets restarted or all the RS gets restarted at the same time.(before assignRootandMeta).

      LOG.info(message);
            if (existingServer.getStartcode() < serverName.getStartcode()) {
              LOG.info("Triggering server recovery; existingServer " +
                existingServer + " looks stale, new server:" + serverName);
              expireServer(existingServer);
            }
      

      If another RS is brought up then the cluster comes back to normalcy.

      May be a very corner case.

        Attachments

        1. HBASE-5916_92.patch
          11 kB
          rajeshbabu
        2. HBASE-5916_94.patch
          10 kB
          rajeshbabu
        3. HBASE-5916_trunk_1.patch
          11 kB
          ramkrishna.s.vasudevan
        4. HBASE-5916_trunk_1.patch
          11 kB
          ramkrishna.s.vasudevan
        5. HBASE-5916_trunk_2.patch
          11 kB
          ramkrishna.s.vasudevan
        6. HBASE-5916_trunk_3.patch
          12 kB
          ramkrishna.s.vasudevan
        7. HBASE-5916_trunk_4.patch
          7 kB
          ramkrishna.s.vasudevan
        8. HBASE-5916_trunk_v5.patch
          8 kB
          rajeshbabu
        9. HBASE-5916_trunk_v6.patch
          10 kB
          rajeshbabu
        10. HBASE-5916_trunk_v7.patch
          10 kB
          rajeshbabu
        11. HBASE-5916_trunk_v8.patch
          10 kB
          rajeshbabu
        12. HBASE-5916_trunk_v9.patch
          10 kB
          ramkrishna.s.vasudevan
        13. HBASE-5916_trunk.patch
          10 kB
          ramkrishna.s.vasudevan
        14. HBASE-5916v8.patch
          5 kB
          chunhui shen

          Issue Links

            Activity

              People

              • Assignee:
                ram_krish ramkrishna.s.vasudevan
                Reporter:
                ram_krish ramkrishna.s.vasudevan
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: