Hadoop Common
  1. Hadoop Common
  2. HADOOP-7949

Updated maxIdleTime default in the code to match core-default.xml

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 0.21.0, 0.22.0, 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: ipc
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-2909 intended to set the server max idle time for a connection to twice the client value. ("The server-side max idle time should be greater than the client-side max idle time, for example, twice of the client-side max idle time.") This way when a server times out a connection it's due a crashed client and not an inactive client so we don't close client connections with outstanding requests (by setting 2x the client value on the server side the client should time out the connection first).

      Looks like there was a typo in the patch and it set the default value to 1/5th the client value, instead of the intended 2x.

      hadoop2 (pre-HADOOP-4687)$ git reset --hard 6fa4597e
      hadoop2 (pre-HADOOP-4687)$ grep -r ipc.client.connection.maxidletime . 
      ./src/core/org/apache/hadoop/ipc/Client.java:      conf.getInt("ipc.client.connection.maxidletime", 10000); //10s
      ./src/core/org/apache/hadoop/ipc/Server.java:    this.maxIdleTime = 2*conf.getInt("ipc.client.connection.maxidletime", 1000);
      
      1. hadoop-7949.txt
        11 kB
        Eli Collins
      2. hadoop-7949.txt
        11 kB
        Eli Collins

        Issue Links

          Activity

          Eli Collins created issue -
          Eli Collins made changes -
          Field Original Value New Value
          Link This issue relates to HADOOP-2909 [ HADOOP-2909 ]
          Hide
          Eli Collins added a comment -

          Patch attached. Defaults maxidletime in Server.java to 10000 instead of 1000, and updates the various hard-coded ipc keys and defaults in Client and Server to use the relevant constants to prevent similar future mistakes.

          Show
          Eli Collins added a comment - Patch attached. Defaults maxidletime in Server.java to 10000 instead of 1000, and updates the various hard-coded ipc keys and defaults in Client and Server to use the relevant constants to prevent similar future mistakes.
          Eli Collins made changes -
          Attachment hadoop-7949.txt [ 12509374 ]
          Hide
          Harsh J added a comment -

          +1, the changes look good to me. Thanks for 'constantizing' this!

          Show
          Harsh J added a comment - +1, the changes look good to me. Thanks for 'constantizing' this!
          Harsh J made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Target Version/s 1.1.0, 0.23.1 [ 12316501, 12318884 ] 0.23.1, 1.1.0 [ 12318884, 12316501 ]
          Eli Collins made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 1.1.0, 0.23.1 [ 12316501, 12318884 ] 0.23.1, 1.1.0 [ 12318884, 12316501 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12509374/hadoop-7949.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 5 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.ipc.TestSaslRPC

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/495//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/495//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509374/hadoop-7949.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 5 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.ipc.TestSaslRPC +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/495//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/495//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          Todd pointed out ipc.client.connection.maxidletime is set in core-default.xml so the defaults here in the code are actually dead code. Doesn't seem like we should define defaults in two places (the code and -default.xml) I'll file a jira for that.

          Test failure is cut paste error in the test, will fix that.

          Show
          Eli Collins added a comment - Todd pointed out ipc.client.connection.maxidletime is set in core-default.xml so the defaults here in the code are actually dead code. Doesn't seem like we should define defaults in two places (the code and -default.xml) I'll file a jira for that. Test failure is cut paste error in the test, will fix that.
          Eli Collins made changes -
          Link This issue relates to HADOOP-7956 [ HADOOP-7956 ]
          Hide
          Eli Collins added a comment -

          Updated patch, just fixes the copy past error in the test.

          Show
          Eli Collins added a comment - Updated patch, just fixes the copy past error in the test.
          Eli Collins made changes -
          Attachment hadoop-7949.txt [ 12509457 ]
          Eli Collins made changes -
          Summary Server maxIdleTime is way too low Updated maxIdleTime default in the code to match core-default.xml
          Target Version/s 1.1.0, 0.23.1 [ 12316501, 12318884 ] 0.23.1 [ 12318884 ]
          Priority Major [ 3 ] Trivial [ 5 ]
          Eli Collins made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Eli Collins made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12509457/hadoop-7949.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 5 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/496//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/496//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509457/hadoop-7949.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 5 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/496//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/496//console This message is automatically generated.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1569 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1569/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1569 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1569/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1498 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1498/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1498 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1498/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Eli Collins added a comment -

          Thanks for the review Harsh. I've committed this and merged to 23.

          Show
          Eli Collins added a comment - Thanks for the review Harsh. I've committed this and merged to 23.
          Eli Collins made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1518 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1518/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1518 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1518/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #916 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/916/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #916 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/916/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #949 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/949/)
          HADOOP-7949. Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #949 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/949/ ) HADOOP-7949 . Updated maxIdleTime default in the code to match core-default.xml. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1227423 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
          Hide
          Harsh J added a comment -

          Eli,

          Is this pending a 0.23 backport? I do not see a Fix Version(s) set.

          Show
          Harsh J added a comment - Eli, Is this pending a 0.23 backport? I do not see a Fix Version(s) set.
          Hide
          Eli Collins added a comment -

          This was fixed in 23. Updating fixVersion.

          Show
          Eli Collins added a comment - This was fixed in 23. Updating fixVersion.
          Eli Collins made changes -
          Fix Version/s 0.23.1 [ 12318884 ]
          Target Version/s 0.23.1 [ 12318884 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          1h 52m 1 Eli Collins 04/Jan/12 21:01
          Open Open Patch Available Patch Available
          14h 34m 2 Eli Collins 04/Jan/12 21:01
          Patch Available Patch Available Resolved Resolved
          4h 47m 1 Eli Collins 05/Jan/12 01:49
          Resolved Resolved Closed Closed
          60d 59m 1 Arun C Murthy 05/Mar/12 02:48

            People

            • Assignee:
              Eli Collins
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development