HBase
  1. HBase
  2. HBASE-5873

TimeOut Monitor thread should be started after atleast one region server registers.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.90.6
    • Fix Version/s: 0.94.0, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently timeout monitor thread is started even before the region server has registered with the master.
      In timeout monitor we depend on the region server to be online

      boolean allRSsOffline = this.serverManager.getOnlineServersList().
              isEmpty();
      

      Now when the master starts up it sees there are no online servers and hence sets
      allRSsOffline to true.

      setAllRegionServersOffline(allRSsOffline);
      

      So this.allRegionServersOffline is also true.
      By this time an RS has come up,
      Now timeout comes up again (after 10secs) in the next cycle he sees allRSsOffline as false.
      Hence

      else if (this.allRegionServersOffline && !allRSsOffline) {
                  // if some RSs just came back online, we can start the
                  // the assignment right away
                  actOnTimeOut(regionState);
      

      This condition makes him to take action based on timeout.
      Because of this even if one Region assignment of ROOT is going on, this piece of code triggers another assignment and thus we get RegionAlreadyinTransition Exception. Later we need to wait for 30 mins for assigning ROOT itself.

      1. HBASE-5873.patch
        2 kB
        rajeshbabu
      2. 5873-trunk.txt
        2 kB
        Lars Hofhansl

        Activity

        Hide
        rajeshbabu added a comment -

        Attached patch.Please review and provide your comments/suggestions.

        Show
        rajeshbabu added a comment - Attached patch.Please review and provide your comments/suggestions.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12524287/HBASE-5873.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1645//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12524287/HBASE-5873.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1645//console This message is automatically generated.
        Hide
        rajeshbabu added a comment -

        This patch uploaded for 0.94

        Show
        rajeshbabu added a comment - This patch uploaded for 0.94
        Hide
        Lars Hofhansl added a comment -

        Trunk patch for HadoopQA

        Show
        Lars Hofhansl added a comment - Trunk patch for HadoopQA
        Hide
        Lars Hofhansl added a comment -

        Oops, rajesh, we crossed comments.

        Show
        Lars Hofhansl added a comment - Oops, rajesh, we crossed comments.
        Hide
        Ted Yu added a comment -

        +1 if tests pass.

        Show
        Ted Yu added a comment - +1 if tests pass.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12524297/5873-trunk.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1647//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1647//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1647//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12524297/5873-trunk.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1647//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1647//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1647//console This message is automatically generated.
        Hide
        Lars Hofhansl added a comment -

        Looks good, tests pass. +1 from me.

        Show
        Lars Hofhansl added a comment - Looks good, tests pass. +1 from me.
        Hide
        Lars Hofhansl added a comment -

        This change does violate encapsulation a bit.
        I double checked where in the code we create instances of AssignmentManager. Besides the HMaster it is only from tests (and they all pass it's good).

        Show
        Lars Hofhansl added a comment - This change does violate encapsulation a bit. I double checked where in the code we create instances of AssignmentManager. Besides the HMaster it is only from tests (and they all pass it's good).
        Hide
        Lars Hofhansl added a comment -

        Committed to 0.94 and trunk.
        Creating 0.90 and 0.92 patches now.

        Show
        Lars Hofhansl added a comment - Committed to 0.94 and trunk. Creating 0.90 and 0.92 patches now.
        Hide
        Lars Hofhansl added a comment -

        And committed to 0.90 and 0.92

        Show
        Lars Hofhansl added a comment - And committed to 0.90 and 0.92
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2814 (See https://builds.apache.org/job/HBase-TRUNK/2814/)
        HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330551)

        Result = FAILURE
        larsh :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2814 (See https://builds.apache.org/job/HBase-TRUNK/2814/ ) HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330551) Result = FAILURE larsh : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #149 (See https://builds.apache.org/job/HBase-0.94/149/)
        HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330549)

        Result = SUCCESS
        larsh :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #149 (See https://builds.apache.org/job/HBase-0.94/149/ ) HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330549) Result = SUCCESS larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92 #391 (See https://builds.apache.org/job/HBase-0.92/391/)
        HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330558)

        Result = SUCCESS
        larsh :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Show
        Hudson added a comment - Integrated in HBase-0.92 #391 (See https://builds.apache.org/job/HBase-0.92/391/ ) HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330558) Result = SUCCESS larsh : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #184 (See https://builds.apache.org/job/HBase-TRUNK-security/184/)
        HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330551)

        Result = FAILURE
        larsh :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #184 (See https://builds.apache.org/job/HBase-TRUNK-security/184/ ) HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330551) Result = FAILURE larsh : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Hide
        ramkrishna.s.vasudevan added a comment -

        @Lars
        Thanks for taking care of committing to the other branches also.

        Show
        ramkrishna.s.vasudevan added a comment - @Lars Thanks for taking care of committing to the other branches also.
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #22 (See https://builds.apache.org/job/HBase-0.94-security/22/)
        HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330549)

        Result = FAILURE
        larsh :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #22 (See https://builds.apache.org/job/HBase-0.94-security/22/ ) HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330549) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92-security #106 (See https://builds.apache.org/job/HBase-0.92-security/106/)
        HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330558)

        Result = SUCCESS
        larsh :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        Show
        Hudson added a comment - Integrated in HBase-0.92-security #106 (See https://builds.apache.org/job/HBase-0.92-security/106/ ) HBASE-5873 TimeOut Monitor thread should be started after atleast one region server registers. (Revision 1330558) Result = SUCCESS larsh : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java

          People

          • Assignee:
            rajeshbabu
            Reporter:
            ramkrishna.s.vasudevan
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development