Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: scripts
    • Labels:
      None

      Description

      There seems to be some confusion in HADOOP_CLASSPATH vs. HADOOP_USER_CLASSPATH in the code and documentation, especially since HADOOP_USER_CLASSPATH doesn't appear to be wired up. This needs to to get fixed ASAP.

      1. HADOOP-13349.00.patch
        6 kB
        Allen Wittenauer

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment -

          IIRC, the intent was that HADOOP_USER_CLASSPATH would replace HADOOP_CLASSPATH since it was ambiguous as to what HADOOP_CLASSPATH represented. (Ironic!) However, it looks like that work was never completed and forward ports from branch-2 just made things worse.

          We're going to need to untangle this before 3.0.0-alpha1 for sure since that's a core user-facing API. CC: Andrew Wang.

          Show
          aw Allen Wittenauer added a comment - IIRC, the intent was that HADOOP_USER_CLASSPATH would replace HADOOP_CLASSPATH since it was ambiguous as to what HADOOP_CLASSPATH represented. (Ironic!) However, it looks like that work was never completed and forward ports from branch-2 just made things worse. We're going to need to untangle this before 3.0.0-alpha1 for sure since that's a core user-facing API. CC: Andrew Wang .
          Hide
          aw Allen Wittenauer added a comment -

          Oh, right: this was also made worse by YARN-1429, which added YARN_USER_CLASSPATH as an analog to HADOOP_CLASSPATH.

          Show
          aw Allen Wittenauer added a comment - Oh, right: this was also made worse by YARN-1429 , which added YARN_USER_CLASSPATH as an analog to HADOOP_CLASSPATH.
          Hide
          aw Allen Wittenauer added a comment - - edited

          -00:

          • change all instances of HADOOP_USER_CLASSPATH to HADOOP_CLASSPATH

          Hooray! Almost all of these are documentation changes. The fact is, HADOOP_USER_CLASSPATH is a better name. But since Windows isn't changing, it just makes sense to go back to HADOOP_CLASSPATH so that documentation is similar. If anyone thinks HADOOP_USER_CLASSPATH really is the way to go forward, it's no big deal to change it. This patch becomes more of a code change than a documentation change, however.

          It's worth pointing out even the unit tests were written to HADOOP_CLASSPATH.

          Show
          aw Allen Wittenauer added a comment - - edited -00: change all instances of HADOOP_USER_CLASSPATH to HADOOP_CLASSPATH Hooray! Almost all of these are documentation changes. The fact is, HADOOP_USER_CLASSPATH is a better name. But since Windows isn't changing, it just makes sense to go back to HADOOP_CLASSPATH so that documentation is similar. If anyone thinks HADOOP_USER_CLASSPATH really is the way to go forward, it's no big deal to change it. This patch becomes more of a code change than a documentation change, however. It's worth pointing out even the unit tests were written to HADOOP_CLASSPATH.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 13s Maven dependency ordering for branch
          +1 mvninstall 6m 41s trunk passed
          +1 mvnsite 3m 47s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvnsite 3m 30s the patch passed
          +1 shellcheck 0m 13s There were no new shellcheck issues.
          +1 shelldocs 0m 8s There were no new shelldocs issues.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 unit 1m 43s hadoop-common in the patch passed.
          +1 unit 2m 27s hadoop-yarn in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          19m 57s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12816648/HADOOP-13349.00.patch
          JIRA Issue HADOOP-13349
          Optional Tests asflicense mvnsite unit shellcheck shelldocs
          uname Linux 255f652687ed 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a3f93be
          shellcheck v0.4.4
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9937/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9937/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 13s Maven dependency ordering for branch +1 mvninstall 6m 41s trunk passed +1 mvnsite 3m 47s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvnsite 3m 30s the patch passed +1 shellcheck 0m 13s There were no new shellcheck issues. +1 shelldocs 0m 8s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 1m 43s hadoop-common in the patch passed. +1 unit 2m 27s hadoop-yarn in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 19m 57s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12816648/HADOOP-13349.00.patch JIRA Issue HADOOP-13349 Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux 255f652687ed 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a3f93be shellcheck v0.4.4 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9937/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9937/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          So IIUC, the confusion is because we have an option for HADOOP_USER_CLASSPATH_FIRST, but what that does is put the user's HADOOP_CLASSPATH (or YARN_USER_CLASSPATH) first. There is no HADOOP_USER_CLASSPATH, despite the naming of HADOOP_USER_CLASSPATH_FIRST.

          Assuming my interpretation is correct, I'm +1 since these are basically doc changes. Thanks for finding and fixing this Allen!

          Show
          andrew.wang Andrew Wang added a comment - So IIUC, the confusion is because we have an option for HADOOP_USER_CLASSPATH_FIRST, but what that does is put the user's HADOOP_CLASSPATH (or YARN_USER_CLASSPATH) first. There is no HADOOP_USER_CLASSPATH, despite the naming of HADOOP_USER_CLASSPATH_FIRST. Assuming my interpretation is correct, I'm +1 since these are basically doc changes. Thanks for finding and fixing this Allen!
          Hide
          aw Allen Wittenauer added a comment -

          Close enough.

          Thanks. Committing this.

          Show
          aw Allen Wittenauer added a comment - Close enough. Thanks. Committing this.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10063 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10063/)
          HADOOP-13349. HADOOP_CLASSPATH vs HADOOP_USER_CLASSPATH (aw) (aw: rev a0035661c10119f134a6a686c658c098eb2e2ad8)

          • hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh
          • hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md
          • hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10063 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10063/ ) HADOOP-13349 . HADOOP_CLASSPATH vs HADOOP_USER_CLASSPATH (aw) (aw: rev a0035661c10119f134a6a686c658c098eb2e2ad8) hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh

            People

            • Assignee:
              aw Allen Wittenauer
              Reporter:
              aw Allen Wittenauer
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development