Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4521

mapreduce.user.classpath.first incompatibility with 0.20/1.x

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.1, 2.0.0-alpha
    • Fix Version/s: 3.0.0, 2.0.3-alpha, 0.23.5
    • Component/s: mrv2
    • Labels:
      None

      Description

      In Hadoop 0.20 or 1.x, jobs can specify the user's classpath should appear first by setting the property mapreduce.user.classpath.first to true in the job configuration. However in Hadoop 0.23 or 2.x, this has no effect, as the corresponding property there is mapreduce.job.user.classpath.first.

        Activity

        Thomas Graves made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1228 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1228/)
        MAPREDUCE-4521. mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399011)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399011
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1228 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1228/ ) MAPREDUCE-4521 . mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399011) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399011 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1198 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1198/)
        MAPREDUCE-4521. mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399011)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399011
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1198 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1198/ ) MAPREDUCE-4521 . mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399011) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399011 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #407 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/407/)
        svn merge -c 1399011. FIXES: MAPREDUCE-4521. mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399014)

        Result = UNSTABLE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399014
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #407 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/407/ ) svn merge -c 1399011. FIXES: MAPREDUCE-4521 . mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399014) Result = UNSTABLE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399014 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #6 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/6/)
        MAPREDUCE-4521. mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399011)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399011
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #6 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/6/ ) MAPREDUCE-4521 . mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399011) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399011 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #2875 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2875/)
        MAPREDUCE-4521. mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399011)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399011
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #2875 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2875/ ) MAPREDUCE-4521 . mapreduce.user.classpath.first incompatibility with 0.20/1.x (Ravi Prakash via bobby) (Revision 1399011) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399011 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ConfigUtil.java
        Robert Joseph Evans made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 3.0.0 [ 12320355 ]
        Fix Version/s 2.0.3-alpha [ 12323275 ]
        Fix Version/s 0.23.5 [ 12323312 ]
        Resolution Fixed [ 1 ]
        Hide
        Robert Joseph Evans added a comment -

        Thanks Ravi,

        I put this into trunk, branch-2, and branch-0.23

        Show
        Robert Joseph Evans added a comment - Thanks Ravi, I put this into trunk, branch-2, and branch-0.23
        Hide
        Robert Joseph Evans added a comment -

        +1 the patch looks good I'll check it in.

        Show
        Robert Joseph Evans added a comment - +1 the patch looks good I'll check it in.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12548800/MAPREDUCE-4521.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2926//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2926//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548800/MAPREDUCE-4521.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2926//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2926//console This message is automatically generated.
        Ravi Prakash made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Target Version/s 3.0.0, 2.0.3-alpha, 0.23.5 [ 12320355, 12323275, 12323312 ]
        Hide
        Ravi Prakash added a comment -

        I tested that MRApps now picks up the deprecated config too, and treats it same as the new one

        Show
        Ravi Prakash added a comment - I tested that MRApps now picks up the deprecated config too, and treats it same as the new one
        Ravi Prakash made changes -
        Attachment MAPREDUCE-4521.patch [ 12548800 ]
        Hide
        Ravi Prakash added a comment -

        Same patch applies to branch-2, trunk.

        Show
        Ravi Prakash added a comment - Same patch applies to branch-2, trunk.
        Ravi Prakash made changes -
        Field Original Value New Value
        Assignee Ravi Prakash [ raviprak ]
        Jason Lowe created issue -

          People

          • Assignee:
            Ravi Prakash
            Reporter:
            Jason Lowe
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development