Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2904

HDFS jars added incorrectly to yarn classpath

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    1. MR-2904.patch
      0.5 kB
      Sharad Agarwal
    2. MAPREDUCE-2904.patch
      4 kB
      Arun C Murthy

      Activity

      Hide
      Sharad Agarwal added a comment -

      Straightforward fix. I could successfully run random writer with hdfs.

      Show
      Sharad Agarwal added a comment - Straightforward fix. I could successfully run random writer with hdfs.
      Hide
      Arun C Murthy added a comment -

      Updated patch - cleaned out a lot of cruft in bin/yarn including stale refs to ivy stuff from common/hdfs and used java6 ability to pass directory paths to drop loops.

      Show
      Arun C Murthy added a comment - Updated patch - cleaned out a lot of cruft in bin/yarn including stale refs to ivy stuff from common/hdfs and used java6 ability to pass directory paths to drop loops.
      Hide
      Mahadev konar added a comment -

      +1 lgtm

      Show
      Mahadev konar added a comment - +1 lgtm
      Hide
      Arun C Murthy added a comment -

      I just committed this. Thanks Sharad!

      Show
      Arun C Murthy added a comment - I just committed this. Thanks Sharad!
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Common-trunk-Commit #805 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/805/)
      MAPREDUCE-2904. Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy.

      acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954
      Files :

      • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
      Show
      Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #805 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/805/ ) MAPREDUCE-2904 . Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-trunk-Commit #882 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/882/)
      MAPREDUCE-2904. Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy.

      acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954
      Files :

      • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #882 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/882/ ) MAPREDUCE-2904 . Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12492117/MAPREDUCE-2904.patch
      against trunk revision .

      +1 @author. The patch does not contain any @author tags.

      +1 tests included. The patch appears to include 8 new or modified tests.

      -1 javadoc. The javadoc tool appears to have generated 87 warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      -1 findbugs. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      +1 core tests. The patch passed unit tests in hadoop-mapreduce-project.

      +1 contrib tests. The patch passed contrib unit tests.

      Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//testReport/
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html
      Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
      Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12492117/MAPREDUCE-2904.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 87 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/554//console This message is automatically generated.
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Mapreduce-trunk-Commit #815 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/815/)
      MAPREDUCE-2904. Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy.

      acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954
      Files :

      • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
      Show
      Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #815 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/815/ ) MAPREDUCE-2904 . Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-trunk #776 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/776/)
      MAPREDUCE-2904. Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy.

      acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954
      Files :

      • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #776 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/776/ ) MAPREDUCE-2904 . Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Mapreduce-trunk #801 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/801/)
      MAPREDUCE-2904. Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy.

      acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954
      Files :

      • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
      • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
      Show
      Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #801 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/801/ ) MAPREDUCE-2904 . Fixed bin/yarn to correctly include HDFS jars and clean up of stale refs to pre-mavenized Hadoop Common and HDFS. Contributed by Sharad Agarwal and Arun C. Murthy. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1162954 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn

        People

        • Assignee:
          Sharad Agarwal
          Reporter:
          Sharad Agarwal
        • Votes:
          0 Vote for this issue
          Watchers:
          2 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development