Hadoop Common
  1. Hadoop Common
  2. HADOOP-8288

Remove references of mapred.child.ulimit etc. since they are not being used any more

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.2
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: None
    • Labels:
      None

      Description

      Courtesy Philip Su, we found that (mapred.child.ulimit, mapreduce.map.ulimit, mapreduce.reduce.ulimit) were not being used at all. The configuration exists but is never used. Its also mentioned in mapred-default.xml and templates/../mapred-site.xml . Also the method getUlimitMemoryCommand in Shell.java is now useless and can be removed.

      1. HADOOP-8288.branch-0.23.patch
        12 kB
        Ravi Prakash
      2. HADOOP-8288.branch-0.23.patch
        12 kB
        Robert Joseph Evans

        Issue Links

          Activity

          Ravi Prakash created issue -
          Ravi Prakash made changes -
          Field Original Value New Value
          Link This issue relates to MAPREDUCE-4134 [ MAPREDUCE-4134 ]
          Hide
          Ravi Prakash added a comment -

          Linking the JIRA which makes the changes in MapReduce

          Show
          Ravi Prakash added a comment - Linking the JIRA which makes the changes in MapReduce
          Hide
          Ravi Prakash added a comment -

          The patch removes references to ulimit.

          Show
          Ravi Prakash added a comment - The patch removes references to ulimit.
          Ravi Prakash made changes -
          Attachment HADOOP-8288.branch-0.23.patch [ 12522986 ]
          Ravi Prakash made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12522986/HADOOP-8288.branch-0.23.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/862//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522986/HADOOP-8288.branch-0.23.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/862//console This message is automatically generated.
          Hide
          Robert Joseph Evans added a comment -

          Kicking Jenkins

          Show
          Robert Joseph Evans added a comment - Kicking Jenkins
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Robert Joseph Evans made changes -
          Attachment HADOOP-8288.branch-0.23.patch [ 12523203 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12523203/HADOOP-8288.branch-0.23.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/866//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523203/HADOOP-8288.branch-0.23.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/866//console This message is automatically generated.
          Hide
          Ravi Prakash added a comment -

          Hey Bobby! I have no idea what patch is complaining about!

          cp: cannot stat `/home/jenkins/buildSupport/lib/*': No such file or directory

          Could it be because I am removing two files in hadoop-tools ?

          The patch applies just fine. I have to svn rm TestUlimit.java and UlimitApp.java but it works

          Show
          Ravi Prakash added a comment - Hey Bobby! I have no idea what patch is complaining about! cp: cannot stat `/home/jenkins/buildSupport/lib/*': No such file or directory Could it be because I am removing two files in hadoop-tools ? The patch applies just fine. I have to svn rm TestUlimit.java and UlimitApp.java but it works
          Hide
          Robert Joseph Evans added a comment -

          This patch is failing because it makes some modifications to the tools project as well. I ran the tests myself and they pass so +1.

          Show
          Robert Joseph Evans added a comment - This patch is failing because it makes some modifications to the tools project as well. I ran the tests myself and they pass so +1.
          Hide
          Robert Joseph Evans added a comment -

          Thanks Ravi, I just put this into trunk, branch-2, and branch-0.23

          Show
          Robert Joseph Evans added a comment - Thanks Ravi, I just put this into trunk, branch-2, and branch-0.23
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.23.3 [ 12320059 ]
          Fix Version/s 2.0.0 [ 12320352 ]
          Fix Version/s 3.0.0 [ 12320357 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2180 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2180/)
          HADOOP-8288. Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2180 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2180/ ) HADOOP-8288 . Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2106 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2106/)
          HADOOP-8288. Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2106 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2106/ ) HADOOP-8288 . Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2123/)
          HADOOP-8288. Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018)

          Result = ABORTED
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2123/ ) HADOOP-8288 . Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018) Result = ABORTED bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #233 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/233/)
          svn merge -c 1328018 from trunk. FIXES: HADOOP-8288. Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328022)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328022
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #233 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/233/ ) svn merge -c 1328018 from trunk. FIXES: HADOOP-8288 . Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328022) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328022 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1020 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1020/)
          HADOOP-8288. Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1020 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1020/ ) HADOOP-8288 . Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1055 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1055/)
          HADOOP-8288. Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1055 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1055/ ) HADOOP-8288 . Remove references of mapred.child.ulimit etc. since they are not being used any more (Ravi Prakash via bobby) (Revision 1328018) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328018 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/DeprecatedProperties.apt.vm /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestUlimit.java /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/UlimitApp.java
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322473 ]
          Fix Version/s 2.0.0-alpha [ 12320352 ]
          Fix Version/s 3.0.0 [ 12320357 ]

            People

            • Assignee:
              Ravi Prakash
              Reporter:
              Ravi Prakash
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development