Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6619

HADOOP_CLASSPATH is overwritten in MR container

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 2.7.2
    • Fix Version/s: 2.8.0, 2.7.3, 2.6.4
    • Component/s: mrv2
    • Labels:
      None

      Description

      Previously env variable HADOOP_CLASSPAH in MR containers inherit from defaults of the worker node. MAPREDUCE-6454 introduced change to overwrite HADOOP_CLASSPATH completely. This caused regression. We need to add additional entries to HADOOP_CLASSPATH instead of completely replacing it.

        Issue Links

          Activity

          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.
          Hide
          jianhe Jian He added a comment -

          Committed to branch-2, branch-2.8, branch-2.7, branch-2.6, thanks Junping Du !
          Thanks shanyu zhao for reviewing the patch !

          Show
          jianhe Jian He added a comment - Committed to branch-2, branch-2.8, branch-2.7, branch-2.6, thanks Junping Du ! Thanks shanyu zhao for reviewing the patch !
          Hide
          jianhe Jian He added a comment -

          lgtm , committing.

          Show
          jianhe Jian He added a comment - lgtm , committing.
          Hide
          shanyu shanyu zhao added a comment -

          +1 This fixed the regression we've been having. Thanks Junping!

          Show
          shanyu shanyu zhao added a comment - +1 This fixed the regression we've been having. Thanks Junping!
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 37s branch-2 passed
          +1 compile 0m 16s branch-2 passed with JDK v1.8.0_66
          +1 compile 0m 21s branch-2 passed with JDK v1.7.0_91
          +1 checkstyle 0m 17s branch-2 passed
          +1 mvnsite 0m 27s branch-2 passed
          +1 mvneclipse 0m 17s branch-2 passed
          +1 findbugs 1m 1s branch-2 passed
          +1 javadoc 0m 20s branch-2 passed with JDK v1.8.0_66
          +1 javadoc 0m 26s branch-2 passed with JDK v1.7.0_91
          +1 mvninstall 0m 20s the patch passed
          +1 compile 0m 13s the patch passed with JDK v1.8.0_66
          +1 javac 0m 13s the patch passed
          +1 compile 0m 17s the patch passed with JDK v1.7.0_91
          +1 javac 0m 17s the patch passed
          +1 checkstyle 0m 11s the patch passed
          +1 mvnsite 0m 23s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 1s the patch passed
          +1 javadoc 0m 17s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 23s the patch passed with JDK v1.7.0_91
          +1 unit 0m 37s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 44s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          18m 14s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:babe025
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784430/MAPREDUCE-6619-branch-2.patch
          JIRA Issue MAPREDUCE-6619
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 438919761a9d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / ad42229
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6284/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
          Max memory used 74MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6284/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 37s branch-2 passed +1 compile 0m 16s branch-2 passed with JDK v1.8.0_66 +1 compile 0m 21s branch-2 passed with JDK v1.7.0_91 +1 checkstyle 0m 17s branch-2 passed +1 mvnsite 0m 27s branch-2 passed +1 mvneclipse 0m 17s branch-2 passed +1 findbugs 1m 1s branch-2 passed +1 javadoc 0m 20s branch-2 passed with JDK v1.8.0_66 +1 javadoc 0m 26s branch-2 passed with JDK v1.7.0_91 +1 mvninstall 0m 20s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_66 +1 javac 0m 13s the patch passed +1 compile 0m 17s the patch passed with JDK v1.7.0_91 +1 javac 0m 17s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 1s the patch passed +1 javadoc 0m 17s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 23s the patch passed with JDK v1.7.0_91 +1 unit 0m 37s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_66. +1 unit 0m 44s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 18m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:babe025 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784430/MAPREDUCE-6619-branch-2.patch JIRA Issue MAPREDUCE-6619 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 438919761a9d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / ad42229 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6284/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common Max memory used 74MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6284/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          It sounds like MAPREDUCE-6454 only went to branch-2 instead of trunk. Submit a patch to fix the regression on branch-2 so far.

          Show
          djp Junping Du added a comment - It sounds like MAPREDUCE-6454 only went to branch-2 instead of trunk. Submit a patch to fix the regression on branch-2 so far.

            People

            • Assignee:
              djp Junping Du
              Reporter:
              shanyu shanyu zhao
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development