Hadoop Common
  1. Hadoop Common
  2. HADOOP-7949

Updated maxIdleTime default in the code to match core-default.xml

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 0.21.0, 0.22.0, 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: ipc
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-2909 intended to set the server max idle time for a connection to twice the client value. ("The server-side max idle time should be greater than the client-side max idle time, for example, twice of the client-side max idle time.") This way when a server times out a connection it's due a crashed client and not an inactive client so we don't close client connections with outstanding requests (by setting 2x the client value on the server side the client should time out the connection first).

      Looks like there was a typo in the patch and it set the default value to 1/5th the client value, instead of the intended 2x.

      hadoop2 (pre-HADOOP-4687)$ git reset --hard 6fa4597e
      hadoop2 (pre-HADOOP-4687)$ grep -r ipc.client.connection.maxidletime . 
      ./src/core/org/apache/hadoop/ipc/Client.java:      conf.getInt("ipc.client.connection.maxidletime", 10000); //10s
      ./src/core/org/apache/hadoop/ipc/Server.java:    this.maxIdleTime = 2*conf.getInt("ipc.client.connection.maxidletime", 1000);
      
      1. hadoop-7949.txt
        11 kB
        Eli Collins
      2. hadoop-7949.txt
        11 kB
        Eli Collins

        Issue Links

          Activity

          Hide
          Eli Collins added a comment -

          Patch attached. Defaults maxidletime in Server.java to 10000 instead of 1000, and updates the various hard-coded ipc keys and defaults in Client and Server to use the relevant constants to prevent similar future mistakes.

          Show
          Eli Collins added a comment - Patch attached. Defaults maxidletime in Server.java to 10000 instead of 1000, and updates the various hard-coded ipc keys and defaults in Client and Server to use the relevant constants to prevent similar future mistakes.
          Hide
          Harsh J added a comment -

          +1, the changes look good to me. Thanks for 'constantizing' this!

          Show
          Harsh J added a comment - +1, the changes look good to me. Thanks for 'constantizing' this!
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12509374/hadoop-7949.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 5 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.ipc.TestSaslRPC

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/495//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/495//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509374/hadoop-7949.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 5 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.ipc.TestSaslRPC +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/495//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/495//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          Todd pointed out ipc.client.connection.maxidletime is set in core-default.xml so the defaults here in the code are actually dead code. Doesn't seem like we should define defaults in two places (the code and -default.xml) I'll file a jira for that.

          Test failure is cut paste error in the test, will fix that.

          Show
          Eli Collins added a comment - Todd pointed out ipc.client.connection.maxidletime is set in core-default.xml so the defaults here in the code are actually dead code. Doesn't seem like we should define defaults in two places (the code and -default.xml) I'll file a jira for that. Test failure is cut paste error in the test, will fix that.
          Hide
          Eli Collins added a comment -

          Updated patch, just fixes the copy past error in the test.

          Show
          Eli Collins added a comment - Updated patch, just fixes the copy past error in the test.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12509457/hadoop-7949.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 5 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/496//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/496//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509457/hadoop-7949.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 5 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/496//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/496//console This message is automatically generated.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1569 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1569/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1569 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1569/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1498 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1498/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1498 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1498/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Eli Collins added a comment -

          Thanks for the review Harsh. I've committed this and merged to 23.

          Show
          Eli Collins added a comment - Thanks for the review Harsh. I've committed this and merged to 23.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1518 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1518/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1518 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1518/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #916 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/916/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #916 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/916/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #949 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/949/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #949 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/949/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Harsh J added a comment -

          Eli,

          Is this pending a 0.23 backport? I do not see a Fix Version(s) set.

          Show
          Harsh J added a comment - Eli, Is this pending a 0.23 backport? I do not see a Fix Version(s) set.
          Hide
          Eli Collins added a comment -

          This was fixed in 23. Updating fixVersion.

          Show
          Eli Collins added a comment - This was fixed in 23. Updating fixVersion.

            People

            • Assignee:
              Eli Collins
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development