HBase
  1. HBase
  2. HBASE-5618

SplitLogManager - prevent unnecessary attempts to resubmits

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.92.2, 0.94.0
    • Component/s: wal, Zookeeper
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently once a watch fires that the task node has been updated (hearbeated) by the worker, the splitlogmanager still quite some time before it updates the "last heard from" time. This is because the manager currently schedules another getDataSetWatch() and only after that finishes will it update the task's "last heard from" time.

      This leads to a large number of zk-BadVersion warnings when resubmission is continuously attempted and it fails.

      Two changes should be made
      (1) On a resubmission failure because of BadVersion the task's lastUpdate time should get upped.
      (2) The task's lastUpdate time should get upped as soon as the nodeDataChanged() watch fires and without waiting for getDataSetWatch() to complete.

        Activity

        Hide
        Prakash Khemani added a comment -

        update "heartbeat time" as soon as possible and as often as one can.

        Show
        Prakash Khemani added a comment - update "heartbeat time" as soon as possible and as often as one can.
        Hide
        stack added a comment -

        +1 on patch

        Show
        stack added a comment - +1 on patch
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12520020/0001-HBASE-5618-SplitLogManager-prevent-unnecessary-attem.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.master.TestSplitLogManager

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1377//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1377//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1377//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12520020/0001-HBASE-5618-SplitLogManager-prevent-unnecessary-attem.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1377//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1377//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1377//console This message is automatically generated.
        Hide
        stack added a comment -

        That test failure don't look too good.

        Show
        stack added a comment - That test failure don't look too good.
        Hide
        Prakash Khemani added a comment -

        sorry for the test failure. fixed and verified.

        Show
        Prakash Khemani added a comment - sorry for the test failure. fixed and verified.
        Hide
        Prakash Khemani added a comment -

        patch without directory prefixes

        Show
        Prakash Khemani added a comment - patch without directory prefixes
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12521225/0001-HBASE-5618-SplitLogManager-prevent-unnecessary-attem.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.regionserver.TestColumnSeeking

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1380//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1380//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1380//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521225/0001-HBASE-5618-SplitLogManager-prevent-unnecessary-attem.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestColumnSeeking Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1380//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1380//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1380//console This message is automatically generated.
        Hide
        Prakash Khemani added a comment -

        TestColumnSeeking isn't failing for me.

        Show
        Prakash Khemani added a comment - TestColumnSeeking isn't failing for me.
        Hide
        Prakash Khemani added a comment -

        re-attaching the same patch. I had cancelled it by mistake.

        Show
        Prakash Khemani added a comment - re-attaching the same patch. I had cancelled it by mistake.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12521428/0001-HBASE-5618-SplitLogManager-prevent-unnecessary-attem.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1441//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1441//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1441//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521428/0001-HBASE-5618-SplitLogManager-prevent-unnecessary-attem.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1441//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1441//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1441//console This message is automatically generated.
        Hide
        stack added a comment -

        Applied to 0.92, 0.94 and trunk. Thanks for the patch Prakash.

        Show
        stack added a comment - Applied to 0.92, 0.94 and trunk. Thanks for the patch Prakash.
        Hide
        stack added a comment -

        I tried the above failing test locally and it passed for me.

        Show
        stack added a comment - I tried the above failing test locally and it passed for me.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2727 (See https://builds.apache.org/job/HBase-TRUNK/2727/)
        HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310923)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2727 (See https://builds.apache.org/job/HBase-TRUNK/2727/ ) HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310923) Result = SUCCESS stack : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #98 (See https://builds.apache.org/job/HBase-0.94/98/)
        HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310922)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #98 (See https://builds.apache.org/job/HBase-0.94/98/ ) HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310922) Result = SUCCESS stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92 #362 (See https://builds.apache.org/job/HBase-0.92/362/)
        HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310921)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Show
        Hudson added a comment - Integrated in HBase-0.92 #362 (See https://builds.apache.org/job/HBase-0.92/362/ ) HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310921) Result = FAILURE stack : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #162 (See https://builds.apache.org/job/HBase-TRUNK-security/162/)
        HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310923)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #162 (See https://builds.apache.org/job/HBase-TRUNK-security/162/ ) HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310923) Result = FAILURE stack : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #9 (See https://builds.apache.org/job/HBase-0.94-security/9/)
        HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310922)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #9 (See https://builds.apache.org/job/HBase-0.94-security/9/ ) HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310922) Result = SUCCESS stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92-security #105 (See https://builds.apache.org/job/HBase-0.92-security/105/)
        HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310921)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
        Show
        Hudson added a comment - Integrated in HBase-0.92-security #105 (See https://builds.apache.org/job/HBase-0.92-security/105/ ) HBASE-5618 SplitLogManager - prevent unnecessary attempts to resubmits (Revision 1310921) Result = FAILURE stack : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java

          People

          • Assignee:
            Prakash Khemani
            Reporter:
            Prakash Khemani
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development