Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.0, 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      It would be useful for 3rd party apps to know the locations of things when hadoop is running without explicit path env vars set.

      1. HADOOP-12366.00.patch
        5 kB
        Allen Wittenauer
      2. HADOOP-12366.01.patch
        6 kB
        Allen Wittenauer
      3. HADOOP-12366.branch-2.02.patch
        6 kB
        Allen Wittenauer
      4. HADOOP-12366.02.patch
        6 kB
        Allen Wittenauer
      5. HADOOP-12366.branch-2.02.patch
        6 kB
        Allen Wittenauer

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user acmnu closed the pull request at:

          https://github.com/apache/bigtop/pull/288

          Show
          githubbot ASF GitHub Bot added a comment - Github user acmnu closed the pull request at: https://github.com/apache/bigtop/pull/288
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user acmnu opened a pull request:

          https://github.com/apache/bigtop/pull/288

          Bump Hadoop to 2.8.1

          Patches:
          =======
          HADOOP-12366 was ported to 2.8.1
          HADOOP-11628 was removed because it was merged to 2.8

          Build time issues fixed:
          ========================
          Fuse_dfs target dir has been changed in 2.8
          -------------------------------------------
          + cp hadoop-hdfs-project/hadoop-hdfs/target/native/main/native/fuse-dfs/fuse_dfs build/bin/
          cp: cannot stat 'hadoop-hdfs-project/hadoop-hdfs/target/native/main/native/fuse-dfs/fuse_dfs': No such file or directory

          add package goal to mvn
          ----------------------
          Without that goal tar is incomplete:

          + for dir in '$

          {HADOOP_DIR}/{lib,}' '${HDFS_DIR}/{lib,}' '${YARN_DIR}/{lib,}' '${MAPREDUCE_DIR}/{lib,}'
          + '[' -e /code/build/hadoop/rpm/BUILDROOT/hadoop-2.8.1-243.el7.centos.x86_64/usr/lib/hadoop-mapreduce//hadoop-common-2.8.1.jar ']'
          + exit 1
          error: Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install)
          Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install)

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/arenadata/bigtop BIGTOP-2893

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/bigtop/pull/288.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #288

          ----
          commit 009dec7795678ff64a83c72147379d8ced208f52
          Author: Anton Chevychalov <cab@arenadata.io pub@mnu.pp.ru>
          Date: 2017-09-27T11:48:38Z

          Bump Hadoop to 2.8.1

          Patches:
          =======
          HADOOP-12366 was ported to 2.8.1
          HADOOP-11628 was removed because it was merged to 2.8

          Build time issues fixed:
          ========================
          Fuse_dfs target dir has been changed in 2.8
          -------------------------------------------
          + cp hadoop-hdfs-project/hadoop-hdfs/target/native/main/native/fuse-dfs/fuse_dfs build/bin/
          cp: cannot stat 'hadoop-hdfs-project/hadoop-hdfs/target/native/main/native/fuse-dfs/fuse_dfs': No such file or directory

          add package goal to mvn
          ----------------------
          Without that goal tar is incomplete:

          + for dir in '${HADOOP_DIR}

          /

          {lib,}' '${HDFS_DIR}/{lib,}

          ' '$

          {YARN_DIR}

          /

          {lib,}' '${MAPREDUCE_DIR}/{lib,}

          '
          + '[' -e /code/build/hadoop/rpm/BUILDROOT/hadoop-2.8.1-243.el7.centos.x86_64/usr/lib/hadoop-mapreduce//hadoop-common-2.8.1.jar ']'
          + exit 1
          error: Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install)
          Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install)


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user acmnu opened a pull request: https://github.com/apache/bigtop/pull/288 Bump Hadoop to 2.8.1 Patches: ======= HADOOP-12366 was ported to 2.8.1 HADOOP-11628 was removed because it was merged to 2.8 Build time issues fixed: ======================== Fuse_dfs target dir has been changed in 2.8 ------------------------------------------- + cp hadoop-hdfs-project/hadoop-hdfs/target/native/main/native/fuse-dfs/fuse_dfs build/bin/ cp: cannot stat 'hadoop-hdfs-project/hadoop-hdfs/target/native/main/native/fuse-dfs/fuse_dfs': No such file or directory add package goal to mvn ---------------------- Without that goal tar is incomplete: + for dir in '$ {HADOOP_DIR}/{lib,}' '${HDFS_DIR}/{lib,}' '${YARN_DIR}/{lib,}' '${MAPREDUCE_DIR}/{lib,}' + ' [' -e /code/build/hadoop/rpm/BUILDROOT/hadoop-2.8.1-243.el7.centos.x86_64/usr/lib/hadoop-mapreduce//hadoop-common-2.8.1.jar '] ' + exit 1 error: Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install) Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install) You can merge this pull request into a Git repository by running: $ git pull https://github.com/arenadata/bigtop BIGTOP-2893 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/288.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #288 ---- commit 009dec7795678ff64a83c72147379d8ced208f52 Author: Anton Chevychalov <cab@arenadata.io pub@mnu.pp.ru> Date: 2017-09-27T11:48:38Z Bump Hadoop to 2.8.1 Patches: ======= HADOOP-12366 was ported to 2.8.1 HADOOP-11628 was removed because it was merged to 2.8 Build time issues fixed: ======================== Fuse_dfs target dir has been changed in 2.8 ------------------------------------------- + cp hadoop-hdfs-project/hadoop-hdfs/target/native/main/native/fuse-dfs/fuse_dfs build/bin/ cp: cannot stat 'hadoop-hdfs-project/hadoop-hdfs/target/native/main/native/fuse-dfs/fuse_dfs': No such file or directory add package goal to mvn ---------------------- Without that goal tar is incomplete: + for dir in '${HADOOP_DIR} / {lib,}' '${HDFS_DIR}/{lib,} ' '$ {YARN_DIR} / {lib,}' '${MAPREDUCE_DIR}/{lib,} ' + ' [' -e /code/build/hadoop/rpm/BUILDROOT/hadoop-2.8.1-243.el7.centos.x86_64/usr/lib/hadoop-mapreduce//hadoop-common-2.8.1.jar '] ' + exit 1 error: Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install) Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install)
          Hide
          aw Allen Wittenauer added a comment -

          I'm going to revert this from branch-2 & 2.8 given that HADOOP-12857 is going to make using HADOOP_TOOLS_PATH less desirable. I'll revisit this for branch-2 (but not 2.8) after 12857 is settled.

          Show
          aw Allen Wittenauer added a comment - I'm going to revert this from branch-2 & 2.8 given that HADOOP-12857 is going to make using HADOOP_TOOLS_PATH less desirable. I'll revisit this for branch-2 (but not 2.8) after 12857 is settled.
          Hide
          aw Allen Wittenauer added a comment -

          Unless anyone has any objections, I'll be committing this to branch-2 tomorrow.

          Show
          aw Allen Wittenauer added a comment - Unless anyone has any objections, I'll be committing this to branch-2 tomorrow.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2521 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2521/)
          HADOOP-12366. expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730)

          • hadoop-mapreduce-project/bin/mapred
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2521 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2521/ ) HADOOP-12366 . expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730) hadoop-mapreduce-project/bin/mapred hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs hadoop-yarn-project/hadoop-yarn/bin/yarn
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #582 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/582/)
          HADOOP-12366. expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-mapreduce-project/bin/mapred
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #582 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/582/ ) HADOOP-12366 . expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730) hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-mapreduce-project/bin/mapred
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2581 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2581/)
          HADOOP-12366. expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730)

          • hadoop-mapreduce-project/bin/mapred
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2581 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2581/ ) HADOOP-12366 . expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730) hadoop-mapreduce-project/bin/mapred hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-yarn-project/hadoop-yarn/bin/yarn
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1375 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1375/)
          HADOOP-12366. expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730)

          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          • hadoop-mapreduce-project/bin/mapred
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1375 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1375/ ) HADOOP-12366 . expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730) hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs hadoop-mapreduce-project/bin/mapred
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #641 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/641/)
          HADOOP-12366. expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-mapreduce-project/bin/mapred
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #641 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/641/ ) HADOOP-12366 . expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730) hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-mapreduce-project/bin/mapred hadoop-yarn-project/hadoop-yarn/bin/yarn
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #651 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/651/)
          HADOOP-12366. expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730)

          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-mapreduce-project/bin/mapred
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #651 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/651/ ) HADOOP-12366 . expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730) hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-mapreduce-project/bin/mapred
          Hide
          aw Allen Wittenauer added a comment -

          Wow. Is branch-2 really that broken?

          At least the code I actually changed passed. haha.

          Show
          aw Allen Wittenauer added a comment - Wow. Is branch-2 really that broken? At least the code I actually changed passed. haha.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 shellcheck 0m 8s There were no new shellcheck issues.
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 unit 0m 19s hadoop-common in the patch failed with JDK v1.8.0_60.
          -1 unit 0m 7s hadoop-hdfs in the patch failed with JDK v1.8.0_60.
          -1 unit 0m 11s hadoop-yarn in the patch failed with JDK v1.8.0_60.
          -1 unit 5m 42s hadoop-mapreduce-project in the patch failed with JDK v1.8.0_60.
          -1 unit 0m 6s hadoop-common in the patch failed with JDK v1.7.0_79.
          -1 unit 0m 7s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
          -1 unit 0m 8s hadoop-yarn in the patch failed with JDK v1.7.0_79.
          -1 unit 0m 21s hadoop-mapreduce-project in the patch failed with JDK v1.7.0_79.
          +1 asflicense 0m 25s Patch does not generate ASF License warnings.
          8m 2s



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-07
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771189/HADOOP-12366.branch-2.02.patch
          JIRA Issue HADOOP-12366
          Optional Tests asflicense unit shellcheck
          uname Linux 44cb0d0c9bae 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision branch-2 / 385dcc4
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          shellcheck v0.4.1
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-mapreduce-project-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-mapreduce-project-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project hadoop-yarn-project/hadoop-yarn U: .
          Max memory used 41MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 shellcheck 0m 8s There were no new shellcheck issues. +1 whitespace 0m 0s Patch has no whitespace issues. -1 unit 0m 19s hadoop-common in the patch failed with JDK v1.8.0_60. -1 unit 0m 7s hadoop-hdfs in the patch failed with JDK v1.8.0_60. -1 unit 0m 11s hadoop-yarn in the patch failed with JDK v1.8.0_60. -1 unit 5m 42s hadoop-mapreduce-project in the patch failed with JDK v1.8.0_60. -1 unit 0m 6s hadoop-common in the patch failed with JDK v1.7.0_79. -1 unit 0m 7s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 unit 0m 8s hadoop-yarn in the patch failed with JDK v1.7.0_79. -1 unit 0m 21s hadoop-mapreduce-project in the patch failed with JDK v1.7.0_79. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 8m 2s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-07 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771189/HADOOP-12366.branch-2.02.patch JIRA Issue HADOOP-12366 Optional Tests asflicense unit shellcheck uname Linux 44cb0d0c9bae 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision branch-2 / 385dcc4 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 shellcheck v0.4.1 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-mapreduce-project-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/artifact/patchprocess/patch-unit-hadoop-mapreduce-project-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project hadoop-yarn-project/hadoop-yarn U: . Max memory used 41MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8050/console This message was automatically generated.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8773 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8773/)
          HADOOP-12366. expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730)

          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-mapreduce-project/bin/mapred
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8773 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8773/ ) HADOOP-12366 . expose calculated paths (aw) (aw: rev b76b0ce51e4ea2c3b9ead3cf0313a84fd1363730) hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-mapreduce-project/bin/mapred hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          Hide
          aw Allen Wittenauer added a comment -

          Thanks. I've committed the trunk version.

          Show
          aw Allen Wittenauer added a comment - Thanks. I've committed the trunk version.
          Hide
          vvasudev Varun Vasudev added a comment -

          +1 for the latest patch for trunk. Allen Wittenauer - the branch-2 version hasn't gone through yetus. Do you want to run that through yetus as well?

          Show
          vvasudev Varun Vasudev added a comment - +1 for the latest patch for trunk. Allen Wittenauer - the branch-2 version hasn't gone through yetus. Do you want to run that through yetus as well?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 7s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 shellcheck 0m 9s There were no new shellcheck issues.
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 unit 1m 44s hadoop-common in the patch passed with JDK v1.8.0_60.
          +1 unit 0m 34s hadoop-hdfs in the patch passed with JDK v1.8.0_60.
          +1 unit 0m 53s hadoop-yarn in the patch passed with JDK v1.8.0_60.
          +1 unit 2m 29s hadoop-mapreduce-project in the patch passed with JDK v1.8.0_60.
          +1 unit 1m 41s hadoop-common in the patch passed with JDK v1.7.0_79.
          +1 unit 0m 32s hadoop-hdfs in the patch passed with JDK v1.7.0_79.
          +1 unit 0m 50s hadoop-yarn in the patch passed with JDK v1.7.0_79.
          +1 unit 2m 30s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_79.
          +1 asflicense 0m 25s Patch does not generate ASF License warnings.
          12m 1s



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-04
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770645/HADOOP-12366.02.patch
          JIRA Issue HADOOP-12366
          Optional Tests asflicense unit shellcheck
          uname Linux 7d3a1536d45c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh
          git revision trunk / 5667129
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          shellcheck v0.4.1
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8029/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project hadoop-yarn-project/hadoop-yarn U: .
          Max memory used 88MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8029/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 7s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 shellcheck 0m 9s There were no new shellcheck issues. +1 whitespace 0m 0s Patch has no whitespace issues. +1 unit 1m 44s hadoop-common in the patch passed with JDK v1.8.0_60. +1 unit 0m 34s hadoop-hdfs in the patch passed with JDK v1.8.0_60. +1 unit 0m 53s hadoop-yarn in the patch passed with JDK v1.8.0_60. +1 unit 2m 29s hadoop-mapreduce-project in the patch passed with JDK v1.8.0_60. +1 unit 1m 41s hadoop-common in the patch passed with JDK v1.7.0_79. +1 unit 0m 32s hadoop-hdfs in the patch passed with JDK v1.7.0_79. +1 unit 0m 50s hadoop-yarn in the patch passed with JDK v1.7.0_79. +1 unit 2m 30s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 12m 1s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-04 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770645/HADOOP-12366.02.patch JIRA Issue HADOOP-12366 Optional Tests asflicense unit shellcheck uname Linux 7d3a1536d45c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh git revision trunk / 5667129 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 shellcheck v0.4.1 JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8029/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project hadoop-yarn-project/hadoop-yarn U: . Max memory used 88MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8029/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 6s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 shellcheck 0m 9s There were no new shellcheck issues.
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 unit 1m 44s hadoop-common in the patch passed with JDK v1.8.0_60.
          +1 unit 0m 32s hadoop-hdfs in the patch passed with JDK v1.8.0_60.
          +1 unit 0m 51s hadoop-yarn in the patch passed with JDK v1.8.0_60.
          +1 unit 2m 18s hadoop-mapreduce-project in the patch passed with JDK v1.8.0_60.
          +1 unit 1m 37s hadoop-common in the patch passed with JDK v1.7.0_79.
          +1 unit 0m 30s hadoop-hdfs in the patch passed with JDK v1.7.0_79.
          +1 unit 0m 48s hadoop-yarn in the patch passed with JDK v1.7.0_79.
          +1 unit 2m 19s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_79.
          +1 asflicense 0m 25s Patch does not generate ASF License warnings.
          11m 27s



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-04
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770645/HADOOP-12366.02.patch
          JIRA Issue HADOOP-12366
          Optional Tests asflicense unit shellcheck
          uname Linux 56a13cf577ce 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh
          git revision trunk / 5667129
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          shellcheck v0.4.1
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8027/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project hadoop-yarn-project/hadoop-yarn U: .
          Max memory used 90MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8027/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 6s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 shellcheck 0m 9s There were no new shellcheck issues. +1 whitespace 0m 0s Patch has no whitespace issues. +1 unit 1m 44s hadoop-common in the patch passed with JDK v1.8.0_60. +1 unit 0m 32s hadoop-hdfs in the patch passed with JDK v1.8.0_60. +1 unit 0m 51s hadoop-yarn in the patch passed with JDK v1.8.0_60. +1 unit 2m 18s hadoop-mapreduce-project in the patch passed with JDK v1.8.0_60. +1 unit 1m 37s hadoop-common in the patch passed with JDK v1.7.0_79. +1 unit 0m 30s hadoop-hdfs in the patch passed with JDK v1.7.0_79. +1 unit 0m 48s hadoop-yarn in the patch passed with JDK v1.7.0_79. +1 unit 2m 19s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 11m 27s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-04 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770645/HADOOP-12366.02.patch JIRA Issue HADOOP-12366 Optional Tests asflicense unit shellcheck uname Linux 56a13cf577ce 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh git revision trunk / 5667129 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 shellcheck v0.4.1 JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8027/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project hadoop-yarn-project/hadoop-yarn U: . Max memory used 90MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8027/console This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          -02:

          • trunk version with fixed quoting
          Show
          aw Allen Wittenauer added a comment - -02: trunk version with fixed quoting
          Hide
          aw Allen Wittenauer added a comment -

          -02:

          • fix quoting
          Show
          aw Allen Wittenauer added a comment - -02: fix quoting
          Hide
          aw Allen Wittenauer added a comment -

          I need to remove the quotes.

          Show
          aw Allen Wittenauer added a comment - I need to remove the quotes.
          Hide
          aw Allen Wittenauer added a comment -

          -01:

          • rebased
          Show
          aw Allen Wittenauer added a comment - -01: rebased
          Hide
          aw Allen Wittenauer added a comment -

          Actually, I'm going to wait to rebase until HADOOP-10787 is committed, since this patch is blocked by that patch.

          Show
          aw Allen Wittenauer added a comment - Actually, I'm going to wait to rebase until HADOOP-10787 is committed, since this patch is blocked by that patch.
          Hide
          aw Allen Wittenauer added a comment -

          Thanks. I'll rebase here in a sec.

          FWIW, I never did the research to figure out why back in 0.21 during the project split why the _DIR vars were made relative to their _HOME's. Worse yet, it was never really documented very well that these vars existed/what-they-did, yet I know vendors are (ab)using them. As part of HADOOP-9902, I actually did add some information on what these vars are/do (in addition to a bunch more made during the project split) in hadoop-layout.sh.example.

          Show
          aw Allen Wittenauer added a comment - Thanks. I'll rebase here in a sec. FWIW, I never did the research to figure out why back in 0.21 during the project split why the _DIR vars were made relative to their _HOME's. Worse yet, it was never really documented very well that these vars existed/what-they-did, yet I know vendors are (ab)using them. As part of HADOOP-9902 , I actually did add some information on what these vars are/do (in addition to a bunch more made during the project split) in hadoop-layout.sh.example.
          Hide
          vvasudev Varun Vasudev added a comment -

          Thanks for the patch Allen Wittenauer!

          Couple of things -
          1) The patch doesn't apply cleanly to trunk any more. I see issues with 'hdfs'.
          2.) Some of the variables printed out(such as JAVA_HOME, HADOOP_COMMON_HOME and HADOOP_CONF_DIR) have absolute paths while some(such as HADOOP_COMMON_DIR, HADOOP_COMMON_LIB_JARS_DIR and HADOOP_COMMON_LIB_NATIVE_DIR) have relative paths. Do you think the difference matters or do we expect users to know what the individual variables are referring to?

          The rest of the patch looks good.

          Show
          vvasudev Varun Vasudev added a comment - Thanks for the patch Allen Wittenauer ! Couple of things - 1) The patch doesn't apply cleanly to trunk any more. I see issues with 'hdfs'. 2.) Some of the variables printed out(such as JAVA_HOME, HADOOP_COMMON_HOME and HADOOP_CONF_DIR) have absolute paths while some(such as HADOOP_COMMON_DIR, HADOOP_COMMON_LIB_JARS_DIR and HADOOP_COMMON_LIB_NATIVE_DIR) have relative paths. Do you think the difference matters or do we expect users to know what the individual variables are referring to? The rest of the patch looks good.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 12s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 44s There were no new javac warning messages.
          +1 javadoc 9m 59s There were no new javadoc warning messages.
          +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
          +1 shellcheck 0m 6s There were no new shellcheck (v0.3.3) issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 common tests 22m 45s Tests passed in hadoop-common.
          -1 hdfs tests 163m 5s Tests failed in hadoop-hdfs.
              221m 22s  



          Reason Tests
          Failed unit tests hadoop.hdfs.web.TestWebHDFSOAuth2



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12753871/HADOOP-12366.00.patch
          Optional Tests javadoc javac unit shellcheck
          git revision trunk / d31a41c
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7582/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7582/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7582/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7582/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 12s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 44s There were no new javac warning messages. +1 javadoc 9m 59s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 shellcheck 0m 6s There were no new shellcheck (v0.3.3) issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 common tests 22m 45s Tests passed in hadoop-common. -1 hdfs tests 163m 5s Tests failed in hadoop-hdfs.     221m 22s   Reason Tests Failed unit tests hadoop.hdfs.web.TestWebHDFSOAuth2 Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753871/HADOOP-12366.00.patch Optional Tests javadoc javac unit shellcheck git revision trunk / d31a41c hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7582/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7582/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7582/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7582/console This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          -00

          • add envvars subcommand with some simple echo statements to print the related vars to stdout
          Show
          aw Allen Wittenauer added a comment - -00 add envvars subcommand with some simple echo statements to print the related vars to stdout
          Hide
          aw Allen Wittenauer added a comment - - edited

          Blocking this on HADOOP-10787 since HADOOP_TOOLS_PATH should be cleaned up first.

          Show
          aw Allen Wittenauer added a comment - - edited Blocking this on HADOOP-10787 since HADOOP_TOOLS_PATH should be cleaned up first.

            People

            • Assignee:
              aw Allen Wittenauer
              Reporter:
              aw Allen Wittenauer
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development