HBase
  1. HBase
  2. HBASE-11403

Fix race conditions around Object#notify

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.99.0, 0.98.3
    • Fix Version/s: 0.99.0, 0.98.4
    • Component/s: Client, regionserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We do have some race conditions there. We don't see them fail in the unit tests, because our #wait are bounded. But from a performance point of view, they do occur. I've reviewed them and fix all the issue I found excepted in the AM (haven't reviewed this one, may be it's fine).

      On a perf test, this seems to improve the max latency.

      1. 11403.98.v1.patch
        5 kB
        Nicolas Liochon
      2. 11403.v1.patch
        6 kB
        Nicolas Liochon

        Activity

        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12651975/11403.v1.patch
        against trunk revision .
        ATTACHMENT ID: 12651975

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12651975/11403.v1.patch against trunk revision . ATTACHMENT ID: 12651975 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. -1 findbugs . The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9828//console This message is automatically generated.
        Hide
        stack added a comment -

        duh

        Very nice Nicolas Liochon +1

        Show
        stack added a comment - duh Very nice Nicolas Liochon +1
        Hide
        Nicolas Liochon added a comment -

        Committed to master, patch does not apply to 0.98, specific patch it on its way.

        Show
        Nicolas Liochon added a comment - Committed to master, patch does not apply to 0.98, specific patch it on its way.
        Hide
        Nicolas Liochon added a comment -

        11409.98.v1 is what I committed to 0.98, after having executed successfully the small & medium tests.

        Show
        Nicolas Liochon added a comment - 11409.98.v1 is what I committed to 0.98, after having executed successfully the small & medium tests.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12652180/11403.98.v1.patch
        against trunk revision .
        ATTACHMENT ID: 12652180

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9836//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12652180/11403.98.v1.patch against trunk revision . ATTACHMENT ID: 12652180 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9836//console This message is automatically generated.
        Hide
        Nicolas Liochon added a comment -

        committed to master and 0.98, thanks for the review Stack!

        Show
        Nicolas Liochon added a comment - committed to master and 0.98, thanks for the review Stack!
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #5229 (See https://builds.apache.org/job/HBase-TRUNK/5229/)
        HBASE-11403 Fix race conditions around Object#notify (nkeywal: rev ab72babd97838317fa0a380fc4d49bf2703ad17c)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #5229 (See https://builds.apache.org/job/HBase-TRUNK/5229/ ) HBASE-11403 Fix race conditions around Object#notify (nkeywal: rev ab72babd97838317fa0a380fc4d49bf2703ad17c) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98 #353 (See https://builds.apache.org/job/HBase-0.98/353/)
        HBASE-11403 Fix race conditions around Object#notify (nkeywal: rev 01d5e7626c04315995333cef424cc5ba9cf4be72)

        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98 #353 (See https://builds.apache.org/job/HBase-0.98/353/ ) HBASE-11403 Fix race conditions around Object#notify (nkeywal: rev 01d5e7626c04315995333cef424cc5ba9cf4be72) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #335 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/335/)
        HBASE-11403 Fix race conditions around Object#notify (nkeywal: rev 01d5e7626c04315995333cef424cc5ba9cf4be72)

        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #335 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/335/ ) HBASE-11403 Fix race conditions around Object#notify (nkeywal: rev 01d5e7626c04315995333cef424cc5ba9cf4be72) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
        Hide
        Andrew Purtell added a comment -

        Belated +1, lgtm

        Show
        Andrew Purtell added a comment - Belated +1, lgtm
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.

          People

          • Assignee:
            Nicolas Liochon
            Reporter:
            Nicolas Liochon
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development