HBase
  1. HBase
  2. HBASE-5916

RS restart just before master intialization we make the cluster non operative

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.92.1, 0.94.0
    • Fix Version/s: 0.92.2, 0.94.1, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Consider a case where my master is getting restarted. RS that was alive when the master restart started, gets restarted before the master initializes the ServerShutDownHandler.

      serverShutdownHandlerEnabled = true;
      

      In this case when the RS tries to register with the master, the master will try to expire the server but the server cannot be expired as still the serverShutdownHandler is not enabled.

      This case may happen when i have only one RS gets restarted or all the RS gets restarted at the same time.(before assignRootandMeta).

      LOG.info(message);
            if (existingServer.getStartcode() < serverName.getStartcode()) {
              LOG.info("Triggering server recovery; existingServer " +
                existingServer + " looks stale, new server:" + serverName);
              expireServer(existingServer);
            }
      

      If another RS is brought up then the cluster comes back to normalcy.

      May be a very corner case.

      1. HBASE-5916v8.patch
        5 kB
        chunhui shen
      2. HBASE-5916_trunk.patch
        10 kB
        ramkrishna.s.vasudevan
      3. HBASE-5916_trunk_v9.patch
        10 kB
        ramkrishna.s.vasudevan
      4. HBASE-5916_trunk_v8.patch
        10 kB
        rajeshbabu
      5. HBASE-5916_trunk_v7.patch
        10 kB
        rajeshbabu
      6. HBASE-5916_trunk_v6.patch
        10 kB
        rajeshbabu
      7. HBASE-5916_trunk_v5.patch
        8 kB
        rajeshbabu
      8. HBASE-5916_trunk_4.patch
        7 kB
        ramkrishna.s.vasudevan
      9. HBASE-5916_trunk_3.patch
        12 kB
        ramkrishna.s.vasudevan
      10. HBASE-5916_trunk_2.patch
        11 kB
        ramkrishna.s.vasudevan
      11. HBASE-5916_trunk_1.patch
        11 kB
        ramkrishna.s.vasudevan
      12. HBASE-5916_trunk_1.patch
        11 kB
        ramkrishna.s.vasudevan
      13. HBASE-5916_94.patch
        10 kB
        rajeshbabu
      14. HBASE-5916_92.patch
        11 kB
        rajeshbabu

        Issue Links

          Activity

            People

            • Assignee:
              ramkrishna.s.vasudevan
              Reporter:
              ramkrishna.s.vasudevan
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development