Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3854

Reinstate environment variable tests in TestMiniMRChildTask

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask.

      Description

      MAPREDUCE-3716 reinstated one of the tests in TestMiniMRChildTask, but there are two more which should be run.

      1. MAPREDUCE-3854.patch
        3 kB
        Tom White
      2. MAPREDUCE-3854.patch
        1 kB
        Tom White

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #991 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/991/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #991 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/991/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #197 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/197/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv)
          svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224)

          Result = FAILURE
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #197 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/197/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #169 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/169/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv)
          svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224)

          Result = FAILURE
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #169 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/169/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #956 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/956/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223)

          Result = FAILURE
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #956 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/956/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #552 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/552/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv)
          svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #552 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/552/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #539 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/539/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv)
          svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #539 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/539/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1802 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1802/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1802 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1802/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1727 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1727/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223)

          Result = SUCCESS
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1727 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1727/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #555 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/555/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv)
          svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224)

          Result = FAILURE
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #555 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/555/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) svn merge --ignore-ancestry -c 1244223 ../../trunk/ (Revision 1244224) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244224 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1739 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1739/)
          MAPREDUCE-3854. Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223)

          Result = FAILURE
          vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1739 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1739/ ) MAPREDUCE-3854 . Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. (Tom White via vinodkv) (Revision 1244223) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244223 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
          Hide
          Vinod Kumar Vavilapalli added a comment -

          I just committed this to trunk and branch-0.23. Thanks Tom!

          Show
          Vinod Kumar Vavilapalli added a comment - I just committed this to trunk and branch-0.23. Thanks Tom!
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Looks good. +1. Pushing this in.

          Show
          Vinod Kumar Vavilapalli added a comment - Looks good. +1. Pushing this in.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12514387/MAPREDUCE-3854.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1845//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1845//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514387/MAPREDUCE-3854.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1845//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1845//console This message is automatically generated.
          Hide
          Tom White added a comment -

          Thanks for the explanation, Vinod. I've changed the test to use LANG rather than HOME as an example of an existing variable that can be overwritten.

          Show
          Tom White added a comment - Thanks for the explanation, Vinod. I've changed the test to use LANG rather than HOME as an example of an existing variable that can be overwritten.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Short answer for the HOME issue you pointed out is - no, we aren't supporting it anymore.

          With 1.0, we never fixed TaskTracker to cleanup the environment(MAPREDUCE-103), so things like HOME needed to be supported. In 0.23.*, it is now set by NodeManager after the ends' clean up. So, $HOME already points to the user's home dir, we shouldn't be needing it anymore.

          Show
          Vinod Kumar Vavilapalli added a comment - Short answer for the HOME issue you pointed out is - no, we aren't supporting it anymore. With 1.0, we never fixed TaskTracker to cleanup the environment( MAPREDUCE-103 ), so things like HOME needed to be supported. In 0.23.*, it is now set by NodeManager after the ends' clean up. So, $HOME already points to the user's home dir, we shouldn't be needing it anymore.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          I already created MAPREDUCE-3767 before, closing that as duplicate of this given you are taking this up.

          Show
          Vinod Kumar Vavilapalli added a comment - I already created MAPREDUCE-3767 before, closing that as duplicate of this given you are taking this up.
          Hide
          Tom White added a comment -

          This patch adds the tests back, but they fail on the line that checks that HOME is set to /tmp (line 223, 273). Is it no longer possible to override HOME or is this exposing a bug?

          Show
          Tom White added a comment - This patch adds the tests back, but they fail on the line that checks that HOME is set to /tmp (line 223, 273). Is it no longer possible to override HOME or is this exposing a bug?

            People

            • Assignee:
              Tom White
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development