HBase
  1. HBase
  2. HBASE-8581

rpc refactor dropped passing the operation timeout through to the rpcclient

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.95.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      jd was poking and noticed that we were not passing the operation timeout down as rpc timeout as we used to in 0.94. Let me fix.

      1. 8581.txt
        13 kB
        stack
      2. 8581v3.txt
        12 kB
        stack

        Activity

        Hide
        stack added a comment -

        Here is a patch to restore what we had in 0.94 passing down an operation timeout.

          1 In HTable, allow setting  an operation timeout on meta.  Needed testing.
          2
          3 In ServerCallable, do a little refactor so cleaner around checking for timeout.
          4
          5 In RpcClient, add back the getRpcTimeout that takes a default timeout and
          6 returns minimum of default rpc timeout and whatever is set in the thread local.
          7 Call this new method on setup of a 'Service' the first time (In 0.94 it was
          8 called when we set up a new proxy, which equates to about the same thing).
          9
         10 In TestClientNoCluster, add a test that verifies we get SocketTimeoutException
         11 when the operation timeout is reached rather than a retries exhausted
         12 exception which is what we were getting when before this patch.
         13
         14 Add a log4j.properties to hbase-client so we get a bit of logging.
        
        Show
        stack added a comment - Here is a patch to restore what we had in 0.94 passing down an operation timeout. 1 In HTable, allow setting an operation timeout on meta. Needed testing. 2 3 In ServerCallable, do a little refactor so cleaner around checking for timeout. 4 5 In RpcClient, add back the getRpcTimeout that takes a default timeout and 6 returns minimum of default rpc timeout and whatever is set in the thread local. 7 Call this new method on setup of a 'Service' the first time (In 0.94 it was 8 called when we set up a new proxy, which equates to about the same thing). 9 10 In TestClientNoCluster, add a test that verifies we get SocketTimeoutException 11 when the operation timeout is reached rather than a retries exhausted 12 exception which is what we were getting when before this patch. 13 14 Add a log4j.properties to hbase-client so we get a bit of logging.
        Hide
        stack added a comment -

        Looking at 0.94, scanners have different timeout mechanism than this one.

        Show
        stack added a comment - Looking at 0.94, scanners have different timeout mechanism than this one.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12583917/8581.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5759//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12583917/8581.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5759//console This message is automatically generated.
        Hide
        Jean-Daniel Cryans added a comment -

        That's a sweet test Stack, I would just add a fail() in RetriesExhaustedException to make it easier to debug. Another nit, duration is not super descriptive and it's the duration of a single call and there could be many of them. Someone could think it's the duration of the whole ServerCallable.

        Show
        Jean-Daniel Cryans added a comment - That's a sweet test Stack, I would just add a fail() in RetriesExhaustedException to make it easier to debug. Another nit, duration is not super descriptive and it's the duration of a single call and there could be many of them. Someone could think it's the duration of the whole ServerCallable .
        Hide
        stack added a comment -

        JD's feedback.

        Show
        stack added a comment - JD's feedback.
        Hide
        Jean-Daniel Cryans added a comment -

        +1

        Show
        Jean-Daniel Cryans added a comment - +1
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12583923/8581v3.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.client.TestHCM

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12583923/8581v3.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestHCM Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5761//console This message is automatically generated.
        Hide
        stack added a comment -

        Committed to 0.95 and 0.98.

        Show
        stack added a comment - Committed to 0.95 and 0.98.
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95 #208 (See https://builds.apache.org/job/hbase-0.95/208/)
        HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484941)
        HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484939)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/branches/0.95/hbase-client/src/test/resources/log4j.properties

        stack :
        Files :

        • /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
        • /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java
        • /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
        • /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
        • /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
        Show
        Hudson added a comment - Integrated in hbase-0.95 #208 (See https://builds.apache.org/job/hbase-0.95/208/ ) HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484941) HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484939) Result = SUCCESS stack : Files : /hbase/branches/0.95/hbase-client/src/test/resources/log4j.properties stack : Files : /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #4135 (See https://builds.apache.org/job/HBase-TRUNK/4135/)
        HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484942)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
        • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java
        • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
        • /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
        • /hbase/trunk/hbase-client/src/test/resources/log4j.properties
        • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #4135 (See https://builds.apache.org/job/HBase-TRUNK/4135/ ) HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484942) Result = FAILURE stack : Files : /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java /hbase/trunk/hbase-client/src/test/resources/log4j.properties /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95-on-hadoop2 #107 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/107/)
        HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484941)
        HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484939)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.95/hbase-client/src/test/resources/log4j.properties

        stack :
        Files :

        • /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
        • /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java
        • /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
        • /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
        • /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
        Show
        Hudson added a comment - Integrated in hbase-0.95-on-hadoop2 #107 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/107/ ) HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484941) HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484939) Result = FAILURE stack : Files : /hbase/branches/0.95/hbase-client/src/test/resources/log4j.properties stack : Files : /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #538 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/538/)
        HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484942)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
        • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java
        • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java
        • /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
        • /hbase/trunk/hbase-client/src/test/resources/log4j.properties
        • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #538 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/538/ ) HBASE-8581 rpc refactor dropped passing the operation timeout through to the rpcclient (Revision 1484942) Result = FAILURE stack : Files : /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java /hbase/trunk/hbase-client/src/test/resources/log4j.properties /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java

          People

          • Assignee:
            stack
            Reporter:
            stack
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development