HBase
  1. HBase
  2. HBASE-8952

Missing error handling can cause RegionServer RPC thread to busy loop forever

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.95.2
    • Component/s: IPC/RPC
    • Labels:
      None

      Description

      If the connection to the client is closed unexpectedly and "at the wrong time", the code will attempt to keep reading from the socket in a busy loop.

      This bug seems to be present in all released versions of HBase, including the tip of the 0.94 and 0.95 branches, however I only ran into it while porting AsyncHBase to 0.95

      1. HBASE-8952.patch
        0.7 kB
        Benoit Sigoure
      2. 8952.094.txt
        4 kB
        stack

        Activity

        Benoit Sigoure created issue -
        Hide
        Benoit Sigoure added a comment -

        Patch to fix the issue in the 0.95 branch.

        Show
        Benoit Sigoure added a comment - Patch to fix the issue in the 0.95 branch.
        Benoit Sigoure made changes -
        Field Original Value New Value
        Attachment HBASE-8952.patch [ 12592386 ]
        Benoit Sigoure made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Benoit Sigoure added a comment -

        I attached a patch that fixes the issue for me in 0.95 – the patch would need to be ported to other branches as well as to the secure RPC implementation, which is separate in 0.94 (ipc/SecureServer.java).

        Show
        Benoit Sigoure added a comment - I attached a patch that fixes the issue for me in 0.95 – the patch would need to be ported to other branches as well as to the secure RPC implementation, which is separate in 0.94 ( ipc/SecureServer.java ).
        Benoit Sigoure made changes -
        Description This bug seems to be present in all released versions of HBase, including the tip of the 0.94 and 0.95 branches. If the connection to the client is closed unexpectedly and "at the wrong time", the code will attempt to keep reading from the socket in a busy loop.

        This bug seems to be present in all released versions of HBase, including the tip of the 0.94 and 0.95 branches, however I only ran into it while porting AsyncHBase to 0.95
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12592386/HBASE-8952.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12592386/HBASE-8952.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6346//console This message is automatically generated.
        Hide
        stack added a comment -

        Patch for 0.94

        Show
        stack added a comment - Patch for 0.94
        stack made changes -
        Attachment 8952.094.txt [ 12592408 ]
        Hide
        stack added a comment -

        Applied to 0.95 and trunk. Lars Hofhansl You want this in 0.94?

        Thanks for the patch Benoit Sigoure

        Show
        stack added a comment - Applied to 0.95 and trunk. Lars Hofhansl You want this in 0.94? Thanks for the patch Benoit Sigoure
        stack made changes -
        Fix Version/s 0.98.0 [ 12323143 ]
        Fix Version/s 0.95.2 [ 12320040 ]
        Fix Version/s 0.94.10 [ 12324627 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12592408/8952.094.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6349//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12592408/8952.094.txt against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6349//console This message is automatically generated.
        Hide
        Lars Hofhansl added a comment -

        Yes, but the 0.94 patch looks like a no op.

        Show
        Lars Hofhansl added a comment - Yes, but the 0.94 patch looks like a no op.
        Hide
        stack added a comment -

        hmm... will commit returning the count.

        Show
        stack added a comment - hmm... will commit returning the count.
        Hide
        stack added a comment -

        Forget it. 0.94 doesn't need this patch. Thanks Lars Hofhansl

        Show
        stack added a comment - Forget it. 0.94 doesn't need this patch. Thanks Lars Hofhansl
        Hide
        stack added a comment -

        Resolving. Thanks B.

        Show
        stack added a comment - Resolving. Thanks B.
        stack made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.94.10 [ 12324627 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #4256 (See https://builds.apache.org/job/HBase-TRUNK/4256/)
        HBASE-8952 Missing error handling can cause RegionServer RPC thread to busy loop forever (stack: rev 1503475)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4256 (See https://builds.apache.org/job/HBase-TRUNK/4256/ ) HBASE-8952 Missing error handling can cause RegionServer RPC thread to busy loop forever (stack: rev 1503475) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.95 #327 (See https://builds.apache.org/job/hbase-0.95/327/)
        HBASE-8952 Missing error handling can cause RegionServer RPC thread to busy loop forever (stack: rev 1503473)

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.95 #327 (See https://builds.apache.org/job/hbase-0.95/327/ ) HBASE-8952 Missing error handling can cause RegionServer RPC thread to busy loop forever (stack: rev 1503473) /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #617 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/617/)
        HBASE-8952 Missing error handling can cause RegionServer RPC thread to busy loop forever (stack: rev 1503475)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #617 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/617/ ) HBASE-8952 Missing error handling can cause RegionServer RPC thread to busy loop forever (stack: rev 1503475) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.95-on-hadoop2 #180 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/180/)
        HBASE-8952 Missing error handling can cause RegionServer RPC thread to busy loop forever (stack: rev 1503473)

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.95-on-hadoop2 #180 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/180/ ) HBASE-8952 Missing error handling can cause RegionServer RPC thread to busy loop forever (stack: rev 1503473) /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
        stack made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Benoit Sigoure
            Reporter:
            Benoit Sigoure
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development