Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12451

[Branch-2] Setting HADOOP_HOME explicitly should be allowed

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.7.2
    • Component/s: scripts
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-11464 reinstates cygwin support. In the process, it sets HADOOP_HOME explicitly in hadoop-config.sh without checking if it has already been set.

      1. HADOOP-12451-branch-2.1.patch
        0.7 kB
        Karthik Kambatla
      2. HADOOP-12451-branch-2.addendum-1.patch
        0.8 kB
        Karthik Kambatla
      3. hadoop-12451-branch-2.addendum-2.patch
        2 kB
        Karthik Kambatla

        Issue Links

          Activity

          Hide
          kasha Karthik Kambatla added a comment -

          Simple patch that adds a check to see if HADOOP_HOME has already been set, similar to how HADOOP_HDFS_HOME is handled.

          Chris Nauroth - can you please review the patch since you know the latest details on how to ensure it runs with cygwin?

          Show
          kasha Karthik Kambatla added a comment - Simple patch that adds a check to see if HADOOP_HOME has already been set, similar to how HADOOP_HDFS_HOME is handled. Chris Nauroth - can you please review the patch since you know the latest details on how to ensure it runs with cygwin?
          Hide
          kasha Karthik Kambatla added a comment -

          This breaks some of our internal tests. I deployed the change and verified the tests pass.

          Show
          kasha Karthik Kambatla added a comment - This breaks some of our internal tests. I deployed the change and verified the tests pass.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 0m 0s Pre-patch branch-2 compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 release audit 0m 16s The applied patch does not increase the total number of release audit warnings.
          +1 shellcheck 0m 4s There were no new shellcheck (v0.3.3) issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
              0m 31s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764361/HADOOP-12451-branch-2.1.patch
          Optional Tests shellcheck
          git revision branch-2 / d645ee1
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7746/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 0m 0s Pre-patch branch-2 compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 release audit 0m 16s The applied patch does not increase the total number of release audit warnings. +1 shellcheck 0m 4s There were no new shellcheck (v0.3.3) issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace.     0m 31s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764361/HADOOP-12451-branch-2.1.patch Optional Tests shellcheck git revision branch-2 / d645ee1 Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7746/console This message was automatically generated.
          Hide
          atm Aaron T. Myers added a comment -

          +1, the patch looks good to me. Might want to wait for Chris Nauroth to review as well, but the change is so straightforward that that I'm pretty confident it won't break what he was trying to fix in HADOOP-11464.

          Show
          atm Aaron T. Myers added a comment - +1, the patch looks good to me. Might want to wait for Chris Nauroth to review as well, but the change is so straightforward that that I'm pretty confident it won't break what he was trying to fix in HADOOP-11464 .
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks for the review, ATM. I ll go ahead and commit this tomorrow at noon PT. Chris Nauroth - are you able to take a quick look at the patch before then?

          Show
          kasha Karthik Kambatla added a comment - Thanks for the review, ATM. I ll go ahead and commit this tomorrow at noon PT. Chris Nauroth - are you able to take a quick look at the patch before then?
          Hide
          cnauroth Chris Nauroth added a comment -

          Karthik Kambatla and Aaron T. Myers, thanks for the notification. I will review this tomorrow.

          Show
          cnauroth Chris Nauroth added a comment - Karthik Kambatla and Aaron T. Myers , thanks for the notification. I will review this tomorrow.
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 for the patch. I tested with Cygwin, and it's all good. Thanks for the fix, Karthik Kambatla!

          Show
          cnauroth Chris Nauroth added a comment - +1 for the patch. I tested with Cygwin, and it's all good. Thanks for the fix, Karthik Kambatla !
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks Chris. Checking this in.

          Show
          kasha Karthik Kambatla added a comment - Thanks Chris. Checking this in.
          Hide
          kasha Karthik Kambatla added a comment -

          Chris Nauroth - trunk scripts are very different. Will create a JIRA to make sure setting HADOOP_HOME explicitly works in trunk as well.

          Show
          kasha Karthik Kambatla added a comment - Chris Nauroth - trunk scripts are very different. Will create a JIRA to make sure setting HADOOP_HOME explicitly works in trunk as well.
          Hide
          kasha Karthik Kambatla added a comment - - edited

          The committed patch prevents HADOOP_HOME from being overwritten.

          In addition to this, HADOOP-11464 changes the behavior in yet another way. It actually sets HADOOP_HOME whereas previously it was left empty. We realized we were relying on HADOOP_HOME not being set at all, and it broke a bunch of things.

          Should we change the patch to set HADOOP_HOME only if it is cygwin. Per Allen Wittenauer's comment here, HADOOP_HOME is supported only on Windows.

          Show
          kasha Karthik Kambatla added a comment - - edited The committed patch prevents HADOOP_HOME from being overwritten. In addition to this, HADOOP-11464 changes the behavior in yet another way. It actually sets HADOOP_HOME whereas previously it was left empty. We realized we were relying on HADOOP_HOME not being set at all, and it broke a bunch of things. Should we change the patch to set HADOOP_HOME only if it is cygwin. Per Allen Wittenauer 's comment here , HADOOP_HOME is supported only on Windows.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Karthik Kambatla, haven't looked at it deeply enough it yet, but do we revert this for now or proceed with a fix?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Karthik Kambatla , haven't looked at it deeply enough it yet, but do we revert this for now or proceed with a fix?
          Hide
          kasha Karthik Kambatla added a comment -

          We should do an addendum. I ll try to get to this this week.

          Show
          kasha Karthik Kambatla added a comment - We should do an addendum. I ll try to get to this this week.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Karthik Kambatla, gentle bump again. Seems like a simple patch. I can wait more for a day or two as I am chasing a few other tickets besides this. Tx.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Karthik Kambatla , gentle bump again. Seems like a simple patch. I can wait more for a day or two as I am chasing a few other tickets besides this. Tx.
          Hide
          kasha Karthik Kambatla added a comment -

          Straight-forward addendum patch that sets HADOOP_HOME only when using cygwin. Haven't tested this on cygwin, since I have no access to Windows box.

          Chris Nauroth, Vinod Kumar Vavilapalli - appreciate any help with test and review.

          Show
          kasha Karthik Kambatla added a comment - Straight-forward addendum patch that sets HADOOP_HOME only when using cygwin. Haven't tested this on cygwin, since I have no access to Windows box. Chris Nauroth , Vinod Kumar Vavilapalli - appreciate any help with test and review.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 shellcheck 0m 5s There were no new shellcheck issues.
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 unit 0m 57s hadoop-common in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 46s hadoop-common in the patch passed with JDK v1.7.0_79.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          2m 31s



          Subsystem Report/Notes
          Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-04
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770653/HADOOP-12451-branch-2.addendum-1.patch
          JIRA Issue HADOOP-12451
          Optional Tests asflicense unit shellcheck
          uname Linux dfe94c1ea082 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh
          git revision branch-2 / 81f7e8a
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          shellcheck v0.4.1
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8030/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Max memory used 42MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8030/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 shellcheck 0m 5s There were no new shellcheck issues. +1 whitespace 0m 0s Patch has no whitespace issues. +1 unit 0m 57s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 0m 46s hadoop-common in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 2m 31s Subsystem Report/Notes Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-04 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770653/HADOOP-12451-branch-2.addendum-1.patch JIRA Issue HADOOP-12451 Optional Tests asflicense unit shellcheck uname Linux dfe94c1ea082 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh git revision branch-2 / 81f7e8a Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 shellcheck v0.4.1 JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8030/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 42MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8030/console This message was automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Karthik Kambatla, looked at the original patch for HADOOP-11464, I think there are more bugs:

          • We should also fix the following line?
            +HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.home.dir=$HADOOP_HOME"

            If it isn't cygwin, it is supposed to point hadoop.home.dir to HADOOP_PREFIX as it was doing before.

          • The same fix (your patch and the change I suggest above) definitely needs to also happen in hadoop-yarn-project/hadoop-yarn/bin/yarn?
          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Karthik Kambatla , looked at the original patch for HADOOP-11464 , I think there are more bugs: We should also fix the following line? +HADOOP_OPTS= "$HADOOP_OPTS -Dhadoop.home.dir=$HADOOP_HOME" If it isn't cygwin, it is supposed to point hadoop.home.dir to HADOOP_PREFIX as it was doing before. The same fix (your patch and the change I suggest above) definitely needs to also happen in hadoop-yarn-project/hadoop-yarn/bin/yarn?
          Hide
          kasha Karthik Kambatla added a comment -

          Looking into this, now.

          Show
          kasha Karthik Kambatla added a comment - Looking into this, now.
          Hide
          cnauroth Chris Nauroth added a comment -

          Just as a heads-up, I will be unavailable Wednesday, 11/11 through Monday, 11/16. If you're relying on me for Cygwin testing, then I would need to do it either before 11/11 or after 11/16.

          Show
          cnauroth Chris Nauroth added a comment - Just as a heads-up, I will be unavailable Wednesday, 11/11 through Monday, 11/16. If you're relying on me for Cygwin testing, then I would need to do it either before 11/11 or after 11/16.
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks for pointing the issues out, Vinod. Updated patch addresses them.

          Show
          kasha Karthik Kambatla added a comment - Thanks for pointing the issues out, Vinod. Updated patch addresses them.
          Hide
          kasha Karthik Kambatla added a comment -

          Brought up a pseudo-distributed cluster on mac, and ran a few MR jobs. Chris Nauroth - could you take it for a spin? If the patch needs any further fixes, I should be able to re-iterate later today. Thanks.

          Show
          kasha Karthik Kambatla added a comment - Brought up a pseudo-distributed cluster on mac, and ran a few MR jobs. Chris Nauroth - could you take it for a spin? If the patch needs any further fixes, I should be able to re-iterate later today. Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 7s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 shellcheck 0m 5s There were no new shellcheck issues.
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 unit 0m 16s hadoop-common in the patch failed with JDK v1.8.0_60.
          -1 unit 0m 7s hadoop-yarn in the patch failed with JDK v1.8.0_60.
          -1 unit 0m 5s hadoop-common in the patch failed with JDK v1.7.0_79.
          -1 unit 0m 7s hadoop-yarn in the patch failed with JDK v1.7.0_79.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          1m 20s



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771405/hadoop-12451-branch-2.addendum-2.patch
          JIRA Issue HADOOP-12451
          Optional Tests asflicense unit shellcheck
          uname Linux 798377fa593e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision branch-2 / a4ff03b
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          shellcheck v0.4.1
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn U: .
          Max memory used 32MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 7s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 shellcheck 0m 5s There were no new shellcheck issues. +1 whitespace 0m 0s Patch has no whitespace issues. -1 unit 0m 16s hadoop-common in the patch failed with JDK v1.8.0_60. -1 unit 0m 7s hadoop-yarn in the patch failed with JDK v1.8.0_60. -1 unit 0m 5s hadoop-common in the patch failed with JDK v1.7.0_79. -1 unit 0m 7s hadoop-yarn in the patch failed with JDK v1.7.0_79. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 1m 20s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771405/hadoop-12451-branch-2.addendum-2.patch JIRA Issue HADOOP-12451 Optional Tests asflicense unit shellcheck uname Linux 798377fa593e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision branch-2 / a4ff03b Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 shellcheck v0.4.1 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn U: . Max memory used 32MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8056/console This message was automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          The latest patch looks good to me.

          Do we have to worry about any of the complaints from Jenkins?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - The latest patch looks good to me. Do we have to worry about any of the complaints from Jenkins?
          Hide
          cnauroth Chris Nauroth added a comment -

          Do we have to worry about any of the complaints from Jenkins?

          On trunk, we have the shelltest Maven profile to trigger BATS testing of the bash scripts. However, for a branch-2 patch like this one, that Maven profile doesn't exist, so mvn fails when we try to run it with that profile. We probably need some additional smarts in the Hadoop Yetus personality to avoid doing this. I filed YETUS-172.

          Chris Nauroth - could you take it for a spin?

          Yes, can do. I'll do the Cygwin testing in the next few hours. Thanks!

          Show
          cnauroth Chris Nauroth added a comment - Do we have to worry about any of the complaints from Jenkins? On trunk, we have the shelltest Maven profile to trigger BATS testing of the bash scripts. However, for a branch-2 patch like this one, that Maven profile doesn't exist, so mvn fails when we try to run it with that profile. We probably need some additional smarts in the Hadoop Yetus personality to avoid doing this. I filed YETUS-172 . Chris Nauroth - could you take it for a spin? Yes, can do. I'll do the Cygwin testing in the next few hours. Thanks!
          Hide
          aw Allen Wittenauer added a comment - - edited

          so mvn fails when we try to run it with that profile.

          It's failing because it can't resolve the hadoop-maven-plugins jar for 2.8.0-snapshot. Maven doesn't care about unmatched profiles:

          $ cd hadoop-tools/hadoop-rumen
          $ mvn compile -Pshelltest
          [INFO] Scanning for projects...
          [INFO]                                                                         
          [INFO] ------------------------------------------------------------------------
          [INFO] Building Apache Hadoop Rumen 3.0.0-SNAPSHOT
          [INFO] ------------------------------------------------------------------------
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-rumen ---
          [INFO] Executing tasks
          
          main:
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-rumen ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] skip non existing resourceDirectory /Users/aw/Src/aw-github/hadoop/hadoop-tools/hadoop-rumen/src/main/resources
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-rumen ---
          [INFO] Changes detected - recompiling the module!
          [INFO] Compiling 95 source files to /Users/aw/Src/aw-github/hadoop/hadoop-tools/hadoop-rumen/target/classes
          [WARNING] /Users/aw/Src/aw-github/hadoop/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/LoggedTask.java: Some input files use or override a deprecated API.
          [WARNING] /Users/aw/Src/aw-github/hadoop/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/LoggedTask.java: Recompile with -Xlint:deprecation for details.
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD SUCCESS
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 3.821 s
          [INFO] Finished at: 2015-11-09T12:31:22-08:00
          [INFO] Final Memory: 27M/310M
          [INFO] ------------------------------------------------------------------------
          [WARNING] The requested profile "shelltest" could not be activated because it does not exist.
          $ echo $?
          0
          
          Show
          aw Allen Wittenauer added a comment - - edited so mvn fails when we try to run it with that profile. It's failing because it can't resolve the hadoop-maven-plugins jar for 2.8.0-snapshot. Maven doesn't care about unmatched profiles: $ cd hadoop-tools/hadoop-rumen $ mvn compile -Pshelltest [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Apache Hadoop Rumen 3.0.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-rumen --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-resources-plugin:2.6:resources ( default -resources) @ hadoop-rumen --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/aw/Src/aw-github/hadoop/hadoop-tools/hadoop-rumen/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.1:compile ( default -compile) @ hadoop-rumen --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 95 source files to /Users/aw/Src/aw-github/hadoop/hadoop-tools/hadoop-rumen/target/classes [WARNING] /Users/aw/Src/aw-github/hadoop/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/LoggedTask.java: Some input files use or override a deprecated API. [WARNING] /Users/aw/Src/aw-github/hadoop/hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/LoggedTask.java: Recompile with -Xlint:deprecation for details. [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3.821 s [INFO] Finished at: 2015-11-09T12:31:22-08:00 [INFO] Final Memory: 27M/310M [INFO] ------------------------------------------------------------------------ [WARNING] The requested profile "shelltest" could not be activated because it does not exist. $ echo $? 0
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 for the latest addendum. I tested all CLI entry points on Cygwin, and it worked correctly.

          Show
          cnauroth Chris Nauroth added a comment - +1 for the latest addendum. I tested all CLI entry points on Cygwin, and it worked correctly.
          Hide
          aw Allen Wittenauer added a comment -

          (I hope users aren't setting -Dhadoop.home.dir directly.... Ugh. Looking at the branch-2 shell code gives me a rash.)

          Show
          aw Allen Wittenauer added a comment - (I hope users aren't setting -Dhadoop.home.dir directly.... Ugh. Looking at the branch-2 shell code gives me a rash.)
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Committed the addendum to branch-2 and branch-2.7. Thanks Karthik.

          Tx Chris for the verifications.

          Karthik Kambatla, you mentioned a separate trunk-fix above. If we need it, please file a ticket so it can be tracked. Thanks.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Committed the addendum to branch-2 and branch-2.7. Thanks Karthik. Tx Chris for the verifications. Karthik Kambatla , you mentioned a separate trunk-fix above. If we need it, please file a ticket so it can be tracked. Thanks.
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks Chris and Vinod. HADOOP-12456 tracks this for trunk.

          Show
          kasha Karthik Kambatla added a comment - Thanks Chris and Vinod. HADOOP-12456 tracks this for trunk.

            People

            • Assignee:
              kasha Karthik Kambatla
              Reporter:
              kasha Karthik Kambatla
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development