Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8216

Be able to differentiate Power failures from Rack switch reboot

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.89-fb
    • Component/s: None
    • Labels:
      None

      Description

      The master in 0.89-fb waits for 5-6 mins to check if RS'es become accessible; when it sees a co-related failure such as a rack-switch-reboot.

      The rationale behind doing this is that it is not worth assigning and reassigning regions – causing churn, when the rack switch reboots are expected to heal themselves in 5-6 mins. In earlier deployments, where this feature was not present, we used to find ourselves in a bad situation for 30mins-1hr.

      However, co-related failures also happen when there is a power failure for the rack. These cases take much longer to heal; so waiting for 5-6 mins is a wasted effort.

      The master should be able to differentiate the two scenario, by checking if any of the RS in the rack is able to communicate. Unless all the servers in the rack are unaccessible, we should proceed with reassigning the regions.

        Attachments

        1. hbase-8216.diff
          5 kB
          Amitanand Aiyer

        Issue Links

          Activity

            People

            • Assignee:
              amitanand Amitanand Aiyer
              Reporter:
              amitanand Amitanand Aiyer

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment