Hadoop Common
  1. Hadoop Common
  2. HADOOP-10775

Shell operations to fail with meaningful errors on windows if winutils.exe not found

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: trunk-win, 2.7.1
    • Fix Version/s: 2.8.0
    • Component/s: util
    • Labels:
      None
    • Environment:

      windows

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      If winutils.exe can't be found HADOOP_HOME wrong/unset or other causes, then an error is logged -but when any of the Shell operations are used, an NPE is raised rather than something meaningful.

      The error message at setup time should be preserved and then raised before any attempt to invoke a winutils-driven process made

      1. HADOOP-10775.patch
        22 kB
        Steve Loughran
      2. HADOOP-10775-002.patch
        22 kB
        Steve Loughran
      3. HADOOP-10775-003.patch
        34 kB
        Steve Loughran
      4. HADOOP-10775-004.patch
        67 kB
        Steve Loughran
      5. HADOOP-10775-005.patch
        100 kB
        Steve Loughran
      6. HADOOP-10775-006.patch
        100 kB
        Steve Loughran
      7. HADOOP-10775-007.patch
        100 kB
        Steve Loughran
      8. HADOOP-10775-008.patch
        100 kB
        Steve Loughran
      9. HADOOP-10775-009.patch
        99 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          Steve Loughran added a comment -

          Example log with stack trace; the minicluster is trying to create a symlink by executing the command returned by Shell.getSymlinkCommand(targetPath, linkPath))

          java.lang.NullPointerException: null
          	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010)
          	at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
          	at org.apache.hadoop.util.Shell.run(Shell.java:418)
          	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
          	at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:171)
          	at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:197)
          	at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:209)
          	at org.apache.slider.test.YarnMiniClusterTestBase.createMiniCluster(YarnMiniClusterTestBase.groovy:241)
          	at org.apache.slider.test.YarnZKMiniClusterTestBase.createMiniCluster(YarnZKMiniClusterTestBase.groovy:105)
          	at org.apache.slider.test.YarnZKMiniClusterTestBase.createMiniCluster(YarnZKMiniClusterTestBase.groovy:124)
          	at org.apache.slider.agent.standalone.TestBadAMHeap.testBadAMHeap(TestBadAMHeap.groovy:43)Standard Output2014-07-02 13:44:56,336 [main] ERROR util.Shell (Shell.java:getWinUtilsPath(336)) - Failed to locate the winutils binary in the hadoop binary pathjava.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
          	at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
          	at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
          	at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
          	at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
          	at org.apache.hadoop.yarn.conf.YarnConfiguration.<clinit>(YarnConfiguration.java:517)
          	at org.apache.slider.common.tools.SliderUtils.createConfiguration(SliderUtils.java:413)
          	at org.apache.slider.test.YarnMiniClusterTestBase.<clinit>(YarnMiniClusterTestBase.groovy:87)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:601)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
          	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
          
          Show
          Steve Loughran added a comment - Example log with stack trace; the minicluster is trying to create a symlink by executing the command returned by Shell.getSymlinkCommand(targetPath, linkPath)) java.lang.NullPointerException: null at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010) at org.apache.hadoop.util.Shell.runCommand(Shell.java:445) at org.apache.hadoop.util.Shell.run(Shell.java:418) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650) at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:171) at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:197) at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:209) at org.apache.slider.test.YarnMiniClusterTestBase.createMiniCluster(YarnMiniClusterTestBase.groovy:241) at org.apache.slider.test.YarnZKMiniClusterTestBase.createMiniCluster(YarnZKMiniClusterTestBase.groovy:105) at org.apache.slider.test.YarnZKMiniClusterTestBase.createMiniCluster(YarnZKMiniClusterTestBase.groovy:124) at org.apache.slider.agent.standalone.TestBadAMHeap.testBadAMHeap(TestBadAMHeap.groovy:43)Standard Output2014-07-02 13:44:56,336 [main] ERROR util.Shell (Shell.java:getWinUtilsPath(336)) - Failed to locate the winutils binary in the hadoop binary pathjava.io.IOException: Could not locate executable null \bin\winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318) at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333) at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326) at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76) at org.apache.hadoop.yarn.conf.YarnConfiguration.<clinit>(YarnConfiguration.java:517) at org.apache.slider.common.tools.SliderUtils.createConfiguration(SliderUtils.java:413) at org.apache.slider.test.YarnMiniClusterTestBase.<clinit>(YarnMiniClusterTestBase.groovy:87) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
          Hide
          Steve Loughran added a comment -

          Also, if winutils.exe is on the path, no check is actually made for this. the code in hadoop assumes

          1. that HADOOP_HOME is defined and exists
          2. that $HADOOP_HOME/bin/winutils.exe exists
          Show
          Steve Loughran added a comment - Also, if winutils.exe is on the path, no check is actually made for this. the code in hadoop assumes that HADOOP_HOME is defined and exists that $HADOOP_HOME/bin/winutils.exe exists
          Hide
          Steve Loughran added a comment -

          As well as retaining the original WINUTILS field, which is null if the field could not be retrieved, I've added two methods {{getWindowsU

          1. getWinutilsPathStrict() raises an IOE
          2. getWinutilsPathRTE() raises an RTE: I've inserted this where the method calling it doesn't raise an IOE.

          Both these will throw an exception which nests the exception caught when trying to set up the path; those also include, on windows a link to a new wiki page.

          I've not been through all references to Shell.WINUTILS; the ones where a path is set up but not executed are left alone (e.g. Shell.getSetPermissionCommand(). Instead I fixed Shell Executor to scan command for null entries and fail fast, with better text.

          Irrespective of what jenkins says, this patch needs to be tested against a windows host

          Show
          Steve Loughran added a comment - As well as retaining the original WINUTILS field, which is null if the field could not be retrieved, I've added two methods {{getWindowsU getWinutilsPathStrict() raises an IOE getWinutilsPathRTE() raises an RTE: I've inserted this where the method calling it doesn't raise an IOE. Both these will throw an exception which nests the exception caught when trying to set up the path; those also include, on windows a link to a new wiki page. The wiki entry is [ https://wiki.apache.org/hadoop/WindowsProblems ] It links to a repo where we can collect the windows binaries. Currently [ mine ] I've not been through all references to Shell.WINUTILS; the ones where a path is set up but not executed are left alone (e.g. Shell.getSetPermissionCommand() . Instead I fixed Shell Executor to scan command for null entries and fail fast, with better text. Irrespective of what jenkins says, this patch needs to be tested against a windows host
          Hide
          Steve Loughran added a comment -

          Patch -002 reinstates the construction time warning, but at INFO, not error.

          Show
          Steve Loughran added a comment - Patch -002 reinstates the construction time warning, but at INFO, not error.
          Hide
          Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 23m 57s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          -1 javac 10m 12s The applied patch generated 1 additional warning messages.
          +1 javadoc 11m 40s There were no new javadoc warning messages.
          +1 release audit 0m 32s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 14s The applied patch generated 8 new checkstyle issues (total was 124, now 127).
          -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 37s mvn install still works.
          +1 eclipse:eclipse 0m 40s The patch built with eclipse:eclipse.
          +1 findbugs 5m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 21m 11s Tests failed in hadoop-common.
          +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common.
          +1 yarn tests 7m 36s Tests passed in hadoop-yarn-server-nodemanager.
              87m 48s  



          Reason Tests
          Timed out tests org.apache.hadoop.io.compress.TestCodec



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12755731/HADOOP-10775-002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6955771
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/diffJavacWarnings.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/console

          This message was automatically generated.

          Show
          Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 23m 57s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. -1 javac 10m 12s The applied patch generated 1 additional warning messages. +1 javadoc 11m 40s There were no new javadoc warning messages. +1 release audit 0m 32s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 14s The applied patch generated 8 new checkstyle issues (total was 124, now 127). -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 40s The patch built with eclipse:eclipse. +1 findbugs 5m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 21m 11s Tests failed in hadoop-common. +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common. +1 yarn tests 7m 36s Tests passed in hadoop-yarn-server-nodemanager.     87m 48s   Reason Tests Timed out tests org.apache.hadoop.io.compress.TestCodec Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12755731/HADOOP-10775-002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6955771 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/diffJavacWarnings.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7658/console This message was automatically generated.
          Hide
          Steve Loughran added a comment -

          Checkstyle

          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:52: Line is longer than 80 characters (found 98).
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:312: Line is longer than 80 characters (found 88).
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:352: Line is longer than 80 characters (found 89).
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:420: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:442: Line is longer than 80 characters (found 87).
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:449: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:463: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:829: Line is longer than 80 characters (found 91).
          

          javac

          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java:[103,24] [deprecation] getWinUtilsPath() in Shell has been deprecated
          

          that is: the newly deprecated method

          Show
          Steve Loughran added a comment - Checkstyle ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:52: Line is longer than 80 characters (found 98). ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:312: Line is longer than 80 characters (found 88). ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:352: Line is longer than 80 characters (found 89). ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:420: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:442: Line is longer than 80 characters (found 87). ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:449: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:463: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:829: Line is longer than 80 characters (found 91). javac > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java:[103,24] [deprecation] getWinUtilsPath() in Shell has been deprecated that is: the newly deprecated method
          Hide
          Steve Loughran added a comment -

          Patch -003

          1. use the RTE-raising clause whenever all winutils shell commands
            are constructed
          2. remove obsolete code (mostly symlink related) that only ran pre-java-7
          3. keep checkstyle happy with lines < 80 chars
          Show
          Steve Loughran added a comment - Patch -003 use the RTE-raising clause whenever all winutils shell commands are constructed remove obsolete code (mostly symlink related) that only ran pre-java-7 keep checkstyle happy with lines < 80 chars
          Hide
          Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 58s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
          +1 javac 8m 0s There were no new javac warning messages.
          +1 javadoc 10m 4s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 50s The applied patch generated 4 new checkstyle issues (total was 194, now 190).
          -1 whitespace 0m 3s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 4m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 23m 0s Tests passed in hadoop-common.
          +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common.
          -1 yarn tests 7m 13s Tests failed in hadoop-yarn-server-nodemanager.
              79m 49s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.nodemanager.containermanager.localizer.TestLocalResourcesTrackerImpl
            hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService
            hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12755798/HADOOP-10775-003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6955771
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/console

          This message was automatically generated.

          Show
          Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 58s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 8m 0s There were no new javac warning messages. +1 javadoc 10m 4s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 50s The applied patch generated 4 new checkstyle issues (total was 194, now 190). -1 whitespace 0m 3s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 23m 0s Tests passed in hadoop-common. +1 yarn tests 1m 58s Tests passed in hadoop-yarn-common. -1 yarn tests 7m 13s Tests failed in hadoop-yarn-server-nodemanager.     79m 49s   Reason Tests Failed unit tests hadoop.yarn.server.nodemanager.containermanager.localizer.TestLocalResourcesTrackerImpl   hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService   hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12755798/HADOOP-10775-003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6955771 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7661/console This message was automatically generated.
          Hide
          Chris Nauroth added a comment - - edited

          Hi Steve Loughran. This looks like a nice improvement. I can do some test runs on Windows after we work through some feedback.

          Is it time to remove Shell#isJava7OrAbove instead of stubbing it to return true? With the promotion of the class to Public status, I'd be concerned that other projects could choose to call it and expect an accurate answer.

          Can we change getQualifiedBin and getQualifiedBinPath to private?

          Should the WINUTILS constant be made private? It appears your patch already cleans up nearly all direct references to it.

          Since this patch essentially removes the Java 6 symlink workaround special path, shall we change SymlinkBaseTest and TestSymlinkLocalFS to remove the emulatingSymlinksOnWindows checks?

          Show
          Chris Nauroth added a comment - - edited Hi Steve Loughran . This looks like a nice improvement. I can do some test runs on Windows after we work through some feedback. Is it time to remove Shell#isJava7OrAbove instead of stubbing it to return true ? With the promotion of the class to Public status, I'd be concerned that other projects could choose to call it and expect an accurate answer. Can we change getQualifiedBin and getQualifiedBinPath to private ? Should the WINUTILS constant be made private ? It appears your patch already cleans up nearly all direct references to it. Since this patch essentially removes the Java 6 symlink workaround special path, shall we change SymlinkBaseTest and TestSymlinkLocalFS to remove the emulatingSymlinksOnWindows checks?
          Hide
          Steve Loughran added a comment -

          I was reluctant to pull any methods as there is a risk they are in use elsewhere —this class may have been tagged as internal, but its been used enough (including the DistributedShell) example, that it's in use. Maybe retain + deprecate?

          Since this patch essentially removes the Java 6 symlink workaround special path, shall we change SymlinkBaseTest and TestSymlinkLocalFS to remove the emulatingSymlinksOnWindows checks?

          +1

          Show
          Steve Loughran added a comment - I was reluctant to pull any methods as there is a risk they are in use elsewhere —this class may have been tagged as internal, but its been used enough (including the DistributedShell) example, that it's in use. Maybe retain + deprecate? Since this patch essentially removes the Java 6 symlink workaround special path, shall we change SymlinkBaseTest and TestSymlinkLocalFS to remove the emulatingSymlinksOnWindows checks? +1
          Hide
          Chris Nauroth added a comment -

          Maybe retain + deprecate?

          OK, sounds like a good safe plan.

          Show
          Chris Nauroth added a comment - Maybe retain + deprecate? OK, sounds like a good safe plan.
          Hide
          Steve Loughran added a comment -

          An iteration with

          1. cleanup of the tests: no need to skip symlink tests on windows
          2. small code review and some cleanup of Shell itself (-> SLF4J; why was the AtomicBoolean completed marked as volatile?
          3. review of tests and switch to the exception raising getWinUtilsPathXXX() calls.
          4. Improve NativeLibraryChecker to include text on windows as to why it couldn't be found (and don't look when not on windows)
          5. if you try to get the path when not on windows, an exception gets raised.
          6. explicit calls to probe for the path existing, and to ask for it as a File class.
          Show
          Steve Loughran added a comment - An iteration with cleanup of the tests: no need to skip symlink tests on windows small code review and some cleanup of Shell itself (-> SLF4J; why was the AtomicBoolean completed marked as volatile? review of tests and switch to the exception raising getWinUtilsPathXXX() calls. Improve NativeLibraryChecker to include text on windows as to why it couldn't be found (and don't look when not on windows) if you try to get the path when not on windows, an exception gets raised. explicit calls to probe for the path existing, and to ask for it as a File class.
          Hide
          Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 22m 20s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 7 new or modified test files.
          -1 javac 8m 46s The applied patch generated 8 additional warning messages.
          -1 javadoc 11m 8s The applied patch generated 1 additional warning messages.
          +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 8s The applied patch generated 13 new checkstyle issues (total was 194, now 199).
          -1 whitespace 0m 9s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 37s mvn install still works.
          +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
          +1 findbugs 5m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 7m 13s Tests passed in hadoop-common.
          +1 yarn tests 2m 2s Tests passed in hadoop-yarn-common.
          +1 yarn tests 8m 28s Tests passed in hadoop-yarn-server-nodemanager.
              70m 59s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764270/HADOOP-10775-004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 80d33b5
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/diffJavacWarnings.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/diffJavadocWarnings.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/console

          This message was automatically generated.

          Show
          Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 22m 20s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 7 new or modified test files. -1 javac 8m 46s The applied patch generated 8 additional warning messages. -1 javadoc 11m 8s The applied patch generated 1 additional warning messages. +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 8s The applied patch generated 13 new checkstyle issues (total was 194, now 199). -1 whitespace 0m 9s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 5m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 7m 13s Tests passed in hadoop-common. +1 yarn tests 2m 2s Tests passed in hadoop-yarn-common. +1 yarn tests 8m 28s Tests passed in hadoop-yarn-server-nodemanager.     70m 59s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764270/HADOOP-10775-004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 80d33b5 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html javac https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/diffJavacWarnings.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/diffJavadocWarnings.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/console This message was automatically generated.
          Hide
          Steve Loughran added a comment -

          Checkstyle

          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:52: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:59: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:64: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:69: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:269: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:377: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:465: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:468: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:471: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:527: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:541: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:566: First sentence should end with a period.
          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:571: First sentence should end with a period.
          

          And javac, which is from the new deprecation warnings

          https://builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/diffJavacWarnings.txt
          

          will fix

          Show
          Steve Loughran added a comment - Checkstyle ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:52: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:59: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:64: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:69: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:269: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:377: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:465: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:468: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:471: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:527: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:541: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:566: First sentence should end with a period. ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:571: First sentence should end with a period. And javac, which is from the new deprecation warnings https: //builds.apache.org/job/PreCommit-HADOOP-Build/7735/artifact/patchprocess/diffJavacWarnings.txt will fix
          Hide
          Steve Loughran added a comment -

          javac warnings, from uses of the newly deprecated stuff. I thought I'd purged them all, but one more run through clearly needed so I can be confident that all outstanding uses are needed

          17a18
          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java:[78,16] [deprecation] isJava7OrAbove() in Shell has been deprecated
          28a30
          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java:[131,13] [deprecation] WINUTILS in Shell has been deprecated
          209c211
          < [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java:[1099,33] [deprecation] containsString(String) in JUnitMatchers has been deprecated
          ---
          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java:[1098,33] [deprecation] containsString(String) in JUnitMatchers has been deprecated
          210a213,218
          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[46,38] [deprecation] WINUTILS in Shell has been deprecated
          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[57,20] [deprecation] WINUTILS in Shell has been deprecated
          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[71,20] [deprecation] WINUTILS in Shell has been deprecated
          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[88,31] [deprecation] WINUTILS in Shell has been deprecated
          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[102,31] [deprecation] WINUTILS in Shell has been deprecated
          > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[125,14] [deprecation] WINUTILS in Shell has been deprecated
          
          Show
          Steve Loughran added a comment - javac warnings, from uses of the newly deprecated stuff. I thought I'd purged them all, but one more run through clearly needed so I can be confident that all outstanding uses are needed 17a18 > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java:[78,16] [deprecation] isJava7OrAbove() in Shell has been deprecated 28a30 > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java:[131,13] [deprecation] WINUTILS in Shell has been deprecated 209c211 < [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java:[1099,33] [deprecation] containsString( String ) in JUnitMatchers has been deprecated --- > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java:[1098,33] [deprecation] containsString( String ) in JUnitMatchers has been deprecated 210a213,218 > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[46,38] [deprecation] WINUTILS in Shell has been deprecated > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[57,20] [deprecation] WINUTILS in Shell has been deprecated > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[71,20] [deprecation] WINUTILS in Shell has been deprecated > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[88,31] [deprecation] WINUTILS in Shell has been deprecated > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[102,31] [deprecation] WINUTILS in Shell has been deprecated > [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java:[125,14] [deprecation] WINUTILS in Shell has been deprecated
          Hide
          Steve Loughran added a comment -

          This is a bigger patch which tries to do a full cleanup of shell, as well as setting up for unit tests on the (static) resolution of hadoop home and winutils.

          1. addresses javadoc warnings across Shell
          2. strip trailing spaces
          3. move to SLF4J
          4. factor out inner algorithms for locating/validating hadoop home and winutils static fields into package-scoped methods.
          5. Add tests for the now visible-resolution functions.
          6. fix a bug (credit Andy Seaborne) which would trigger an IndexOutOfBoundsException if HADOOP_HOME or hadoop.home.dir was set to the single quote " character. Now all leading and trailing quotes are stripped; tests validate this.
          7. report the empty "" path to Hadoop home as a specific error, not simply a "relative path supplied".
          8. getWinutilsPath() raises an RTE if called and winutils is null. That includes on Unix, where it is implicitly null.
          9. tag as Deprecated tests which rely on WINUTILS references, even if null.
          10. downgrade any constructor-time warnings about winutils not being found to being at debug level —as now attempts to fetch the path will trigger meaningful exceptions in the codepaths attempting to do so.

          The change in getWinutilsPath() is something which could be flagged as incompatible. There's nothing in the Hadoop codebase which appears to break with this change, all that is happing is that NPEs are being replaced with useful text.

          Show
          Steve Loughran added a comment - This is a bigger patch which tries to do a full cleanup of shell, as well as setting up for unit tests on the (static) resolution of hadoop home and winutils. addresses javadoc warnings across Shell strip trailing spaces move to SLF4J factor out inner algorithms for locating/validating hadoop home and winutils static fields into package-scoped methods. Add tests for the now visible-resolution functions. fix a bug (credit Andy Seaborne ) which would trigger an IndexOutOfBoundsException if HADOOP_HOME or hadoop.home.dir was set to the single quote " character. Now all leading and trailing quotes are stripped; tests validate this. report the empty "" path to Hadoop home as a specific error, not simply a "relative path supplied". getWinutilsPath() raises an RTE if called and winutils is null. That includes on Unix, where it is implicitly null. tag as Deprecated tests which rely on WINUTILS references, even if null. downgrade any constructor-time warnings about winutils not being found to being at debug level —as now attempts to fetch the path will trigger meaningful exceptions in the codepaths attempting to do so. The change in getWinutilsPath() is something which could be flagged as incompatible. There's nothing in the Hadoop codebase which appears to break with this change, all that is happing is that NPEs are being replaced with useful text.
          Hide
          Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 34s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 8 new or modified test files.
          +1 javac 7m 52s There were no new javac warning messages.
          -1 javadoc 10m 7s The applied patch generated 1 additional warning messages.
          -1 release audit 0m 15s The applied patch generated 1 release audit warnings.
          -1 checkstyle 1m 29s The applied patch generated 1 new checkstyle issues (total was 334, now 278).
          -1 whitespace 0m 55s The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 4m 37s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 7m 32s Tests failed in hadoop-common.
          +1 yarn tests 2m 0s Tests passed in hadoop-yarn-common.
          +1 yarn tests 8m 23s Tests passed in hadoop-yarn-server-nodemanager.
              63m 14s  



          Reason Tests
          Failed unit tests hadoop.metrics2.impl.TestGangliaMetrics
            hadoop.ipc.TestIPC



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12764762/HADOOP-10775-005.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / fd026f5
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/diffJavadocWarnings.txt
          Release Audit https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/console

          This message was automatically generated.

          Show
          Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 34s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 8 new or modified test files. +1 javac 7m 52s There were no new javac warning messages. -1 javadoc 10m 7s The applied patch generated 1 additional warning messages. -1 release audit 0m 15s The applied patch generated 1 release audit warnings. -1 checkstyle 1m 29s The applied patch generated 1 new checkstyle issues (total was 334, now 278). -1 whitespace 0m 55s The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 37s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 7m 32s Tests failed in hadoop-common. +1 yarn tests 2m 0s Tests passed in hadoop-yarn-common. +1 yarn tests 8m 23s Tests passed in hadoop-yarn-server-nodemanager.     63m 14s   Reason Tests Failed unit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.ipc.TestIPC Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764762/HADOOP-10775-005.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fd026f5 javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/diffJavadocWarnings.txt Release Audit https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7756/console This message was automatically generated.
          Hide
          Chris Nauroth added a comment -

          Steve Loughran, this is a great clean-up, and the extra test coverage is good too.

          1. Please expand the comment on Shell#isJava7OrAbove to describe why it's hard-coded to return true. Something like "Hadoop starting at version X requires Java 7 or above" would help clarify.
          2. why was the AtomicBoolean completed marked as volatile?

            It seems this was trying to compensate for the fact that the AtomicBoolean was instantiated lazily, and then could have been referenced on a different thread. The methods of AtomicBoolean have ordering guarantees, but there wouldn't have been an ordering guarantee on the reference to the AtomicBoolean. I agree with the change made in your patch to guarantee initialization at Shell construction instead. It's easier to understand.

          3. I saw test failures in TestShell#testGetCheckProcessIsAliveCommand, TestShell#testGetSignalKillCommand and TestContainerLaunch#testWindowsShellScriptBuilderLink when running on Windows. See below. For TestShell, it looks like we'll need to canonicalize the expected paths. I didn't dig deep into the TestContainerLaunch failure yet, but it appears to be connected to the max command line length checks.
            Failed tests: 
              TestShell.testGetCheckProcessIsAliveCommand:203->Assert.assertArrayEquals:280->Assert.assertArrayEquals:265->Assert.internalArrayEquals:473 arrays first differed at element [0]; expected:<...oject\hadoop-common\[..\..\hadoop-common-project\hadoop-common\]target\bin\winutils....> but was:<...oject\hadoop-common\[]target\bin\winutils....>
              TestShell.testGetSignalKillCommand:223->Assert.assertArrayEquals:280->Assert.assertArrayEquals:265->Assert.internalArrayEquals:473 arrays first differed at element [0]; expected:<...oject\hadoop-common\[..\..\hadoop-common-project\hadoop-common\]target\bin\winutils....> but was:<...oject\hadoop-common\[]target\bin\winutils....>
            
            testWindowsShellScriptBuilderLink(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)  Time elapsed: 0.078 sec  <<< FAILURE!
            java.lang.AssertionError: long link was expected to throw
            	at org.junit.Assert.fail(Assert.java:88)
            	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testWindowsShellScriptBuilderLink(TestContainerLaunch.java:1096)
            
          4. On Shell#WINDOWS_MAX_SHELL_LENGHT, JavaDoc is being picky and giving a warning about @deprecated:. According to JavaDoc documentation, the tag must be followed immediately by a space or a newline, so it doesn't like the colon.
          5. There is one more checkstyle warning to fix.

          Test failures in the last Jenkins run were unrelated.

          The change in getWinutilsPath() is something which could be flagged as incompatible. There's nothing in the Hadoop codebase which appears to break with this change, all that is happing is that NPEs are being replaced with useful text.

          IIUC, the concern is that an application previously could have caught the NullPointerException and attempted some kind of fallback option. After this patch, it becomes a RuntimeException, so that error handler wouldn't get triggered. I think it would be unusual for this to be a problem in practice, but if we really want to be conservative, then we could continue throwing NullPointerException, populated with the new text. Of course, it isn't really a null pointer problem, but the text would clarify that. Let me know your thoughts.

          Show
          Chris Nauroth added a comment - Steve Loughran , this is a great clean-up, and the extra test coverage is good too. Please expand the comment on Shell#isJava7OrAbove to describe why it's hard-coded to return true . Something like "Hadoop starting at version X requires Java 7 or above" would help clarify. why was the AtomicBoolean completed marked as volatile? It seems this was trying to compensate for the fact that the AtomicBoolean was instantiated lazily, and then could have been referenced on a different thread. The methods of AtomicBoolean have ordering guarantees, but there wouldn't have been an ordering guarantee on the reference to the AtomicBoolean . I agree with the change made in your patch to guarantee initialization at Shell construction instead. It's easier to understand. I saw test failures in TestShell#testGetCheckProcessIsAliveCommand , TestShell#testGetSignalKillCommand and TestContainerLaunch#testWindowsShellScriptBuilderLink when running on Windows. See below. For TestShell , it looks like we'll need to canonicalize the expected paths. I didn't dig deep into the TestContainerLaunch failure yet, but it appears to be connected to the max command line length checks. Failed tests: TestShell.testGetCheckProcessIsAliveCommand:203->Assert.assertArrayEquals:280->Assert.assertArrayEquals:265->Assert.internalArrayEquals:473 arrays first differed at element [0]; expected:<...oject\hadoop-common\[..\..\hadoop-common-project\hadoop-common\]target\bin\winutils....> but was:<...oject\hadoop-common\[]target\bin\winutils....> TestShell.testGetSignalKillCommand:223->Assert.assertArrayEquals:280->Assert.assertArrayEquals:265->Assert.internalArrayEquals:473 arrays first differed at element [0]; expected:<...oject\hadoop-common\[..\..\hadoop-common-project\hadoop-common\]target\bin\winutils....> but was:<...oject\hadoop-common\[]target\bin\winutils....> testWindowsShellScriptBuilderLink(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch) Time elapsed: 0.078 sec <<< FAILURE! java.lang.AssertionError: long link was expected to throw at org.junit.Assert.fail(Assert.java:88) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testWindowsShellScriptBuilderLink(TestContainerLaunch.java:1096) On Shell#WINDOWS_MAX_SHELL_LENGHT , JavaDoc is being picky and giving a warning about @deprecated: . According to JavaDoc documentation , the tag must be followed immediately by a space or a newline, so it doesn't like the colon. There is one more checkstyle warning to fix. Test failures in the last Jenkins run were unrelated. The change in getWinutilsPath() is something which could be flagged as incompatible. There's nothing in the Hadoop codebase which appears to break with this change, all that is happing is that NPEs are being replaced with useful text. IIUC, the concern is that an application previously could have caught the NullPointerException and attempted some kind of fallback option. After this patch, it becomes a RuntimeException , so that error handler wouldn't get triggered. I think it would be unusual for this to be a problem in practice, but if we really want to be conservative, then we could continue throwing NullPointerException , populated with the new text. Of course, it isn't really a null pointer problem, but the text would clarify that. Let me know your thoughts.
          Hide
          Steve Loughran added a comment -

          Patch -006

          1. WINUTILS_FILE is canonicalized during build up; some logic to handle IOEs raised in getCanonicalFile() and getCanonicalPath -impossible but caught & converted to FNFEs.
          2. tested on windows; all is happy
          3. updated deprecation javadocs
          Show
          Steve Loughran added a comment - Patch -006 WINUTILS_FILE is canonicalized during build up; some logic to handle IOEs raised in getCanonicalFile() and getCanonicalPath -impossible but caught & converted to FNFEs. tested on windows; all is happy updated deprecation javadocs
          Hide
          Steve Loughran added a comment -

          One thing to consider is now that exception raising is delayed until use, there's no guarantee that anything will appear in the logs: its up to the final recipient.

          We have the option of adding a log @ error entry when the first attempt to use the winutils path is made. That guarantees there will be something in the log. All that's needed is an atomic bool with some guard like

          if (!winutilsFailureLogged.getAndSet(true)) log.error(WINUTILS_FAILURE)
          

          It will end up in the server log, but only when it is about to cause a problem

          Show
          Steve Loughran added a comment - One thing to consider is now that exception raising is delayed until use, there's no guarantee that anything will appear in the logs: its up to the final recipient. We have the option of adding a log @ error entry when the first attempt to use the winutils path is made. That guarantees there will be something in the log. All that's needed is an atomic bool with some guard like if (!winutilsFailureLogged.getAndSet( true )) log.error(WINUTILS_FAILURE) It will end up in the server log, but only when it is about to cause a problem
          Hide
          Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765229/HADOOP-10775-006.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 29a582a
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7772/console

          This message was automatically generated.

          Show
          Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765229/HADOOP-10775-006.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 29a582a Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7772/console This message was automatically generated.
          Hide
          Steve Loughran added a comment -

          patch doesn't apply no more. rebasing

          Show
          Steve Loughran added a comment - patch doesn't apply no more. rebasing
          Hide
          Steve Loughran added a comment -

          Patch -007

          1. in sync with trunk
          2. fixes HADOOP-12463
          3. tested on windows and unix
          Show
          Steve Loughran added a comment - Patch -007 in sync with trunk fixes HADOOP-12463 tested on windows and unix
          Hide
          Steve Loughran added a comment -

          -008 patch; 007 without some whitespace stripping by an overzealous IDE (atom on windows)

          Show
          Steve Loughran added a comment - -008 patch; 007 without some whitespace stripping by an overzealous IDE (atom on windows)
          Hide
          Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 20m 9s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 8 new or modified test files.
          +1 javac 8m 7s There were no new javac warning messages.
          +1 javadoc 10m 46s There were no new javadoc warning messages.
          -1 release audit 0m 15s The applied patch generated 1 release audit warnings.
          -1 checkstyle 1m 57s The applied patch generated 1 new checkstyle issues (total was 333, now 275).
          -1 whitespace 0m 33s The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
          +1 findbugs 4m 52s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 6m 37s Tests failed in hadoop-common.
          +1 yarn tests 2m 0s Tests passed in hadoop-yarn-common.
          -1 yarn tests 7m 46s Tests failed in hadoop-yarn-server-nodemanager.
              65m 43s  



          Reason Tests
          Failed unit tests hadoop.metrics2.impl.TestGangliaMetrics
            hadoop.net.TestDNS
            hadoop.yarn.server.nodemanager.TestNodeStatusUpdater
            hadoop.yarn.server.nodemanager.TestNodeManagerResync



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765243/HADOOP-10775-008.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6d5713a
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-server-nodemanager.html
          Release Audit https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/console

          This message was automatically generated.

          Show
          Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 20m 9s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 8 new or modified test files. +1 javac 8m 7s There were no new javac warning messages. +1 javadoc 10m 46s There were no new javadoc warning messages. -1 release audit 0m 15s The applied patch generated 1 release audit warnings. -1 checkstyle 1m 57s The applied patch generated 1 new checkstyle issues (total was 333, now 275). -1 whitespace 0m 33s The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 4m 52s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 6m 37s Tests failed in hadoop-common. +1 yarn tests 2m 0s Tests passed in hadoop-yarn-common. -1 yarn tests 7m 46s Tests failed in hadoop-yarn-server-nodemanager.     65m 43s   Reason Tests Failed unit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.net.TestDNS   hadoop.yarn.server.nodemanager.TestNodeStatusUpdater   hadoop.yarn.server.nodemanager.TestNodeManagerResync Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765243/HADOOP-10775-008.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6d5713a Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-server-nodemanager.html Release Audit https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7773/console This message was automatically generated.
          Hide
          Steve Loughran added a comment -

          checkstyle

          ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:675: First sentence should end with a period.
          

          common test failure

          org.apache.hadoop.net.TestDNS.testLookupWithoutHostsFallback
          
          java.lang.AssertionError: 
          Expected: is "-DUMMY_HOSTNAME"
               but: was "localhost"
          	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
          	at org.junit.Assert.assertThat(Assert.java:865)
          	at org.junit.Assert.assertThat(Assert.java:832)
          	at org.apache.hadoop.net.TestDNS.testLookupWithoutHostsFallback(TestDNS.java:224)
          Standard Output
          
          2015-10-06 21:52:38,228 WARN  net.DNS (DNS.java:getHosts(268)) - Unable to determine hostname for interface lo
          2015-10-06 21:52:38,239 WARN  net.DNS (DNS.java:getHosts(268)) - Unable to determine hostname for interface lo
          2015-10-06 21:52:38,296 INFO  net.TestDNS (TestDNS.java:testRDNS(168)) - Local revers DNS hostname is asf900.gq1.ygridcore.net
          2015-10-06 21:52:38,298 INFO  net.TestDNS (TestDNS.java:testLocalhostResolves(261)) - Localhost IPAddr is localhost/127.0.0.1
          2015-10-06 21:52:38,355 WARN  net.DNS (DNS.java:getHosts(268)) - Unable to determine hostname for interface lo
          2015-10-06 21:52:38,403 WARN  net.DNS (DNS.java:getHosts(268)) - Unable to determine hostname for interface lo
          
          Show
          Steve Loughran added a comment - checkstyle ./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java:675: First sentence should end with a period. common test failure org.apache.hadoop.net.TestDNS.testLookupWithoutHostsFallback java.lang.AssertionError: Expected: is "-DUMMY_HOSTNAME" but: was "localhost" at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20) at org.junit.Assert.assertThat(Assert.java:865) at org.junit.Assert.assertThat(Assert.java:832) at org.apache.hadoop.net.TestDNS.testLookupWithoutHostsFallback(TestDNS.java:224) Standard Output 2015-10-06 21:52:38,228 WARN net.DNS (DNS.java:getHosts(268)) - Unable to determine hostname for interface lo 2015-10-06 21:52:38,239 WARN net.DNS (DNS.java:getHosts(268)) - Unable to determine hostname for interface lo 2015-10-06 21:52:38,296 INFO net.TestDNS (TestDNS.java:testRDNS(168)) - Local revers DNS hostname is asf900.gq1.ygridcore.net 2015-10-06 21:52:38,298 INFO net.TestDNS (TestDNS.java:testLocalhostResolves(261)) - Localhost IPAddr is localhost/127.0.0.1 2015-10-06 21:52:38,355 WARN net.DNS (DNS.java:getHosts(268)) - Unable to determine hostname for interface lo 2015-10-06 21:52:38,403 WARN net.DNS (DNS.java:getHosts(268)) - Unable to determine hostname for interface lo
          Hide
          Steve Loughran added a comment -

          Patch 009; a bit of a simplification

          As getWindowsPath() RTEs if there's no valid path, and getWindowsFile() gives the file value or an FNFE, there's no need for a getWindowsPathStrict(); it just complicates things.

          Moved back to getWindowsPath() everywhere, except when an IOE was really wanted —and there I use getWindowsFile().getCanonicalPath()

          Show
          Steve Loughran added a comment - Patch 009; a bit of a simplification As getWindowsPath() RTEs if there's no valid path, and getWindowsFile() gives the file value or an FNFE, there's no need for a getWindowsPathStrict() ; it just complicates things. Moved back to getWindowsPath() everywhere, except when an IOE was really wanted —and there I use getWindowsFile().getCanonicalPath()
          Hide
          Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 20m 1s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 8 new or modified test files.
          +1 javac 8m 4s There were no new javac warning messages.
          +1 javadoc 10m 27s There were no new javadoc warning messages.
          -1 release audit 0m 18s The applied patch generated 1 release audit warnings.
          +1 checkstyle 2m 15s There were no new checkstyle issues.
          -1 whitespace 0m 31s The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 4m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 7m 29s Tests passed in hadoop-common.
          +1 yarn tests 2m 4s Tests passed in hadoop-yarn-common.
          +1 yarn tests 8m 43s Tests passed in hadoop-yarn-server-nodemanager.
              66m 49s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765651/HADOOP-10775-009.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0841940
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-server-nodemanager.html
          Release Audit https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/patchReleaseAuditProblems.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/console

          This message was automatically generated.

          Show
          Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 20m 1s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 8 new or modified test files. +1 javac 8m 4s There were no new javac warning messages. +1 javadoc 10m 27s There were no new javadoc warning messages. -1 release audit 0m 18s The applied patch generated 1 release audit warnings. +1 checkstyle 2m 15s There were no new checkstyle issues. -1 whitespace 0m 31s The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 7m 29s Tests passed in hadoop-common. +1 yarn tests 2m 4s Tests passed in hadoop-yarn-common. +1 yarn tests 8m 43s Tests passed in hadoop-yarn-server-nodemanager.     66m 49s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765651/HADOOP-10775-009.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0841940 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-server-nodemanager.html Release Audit https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/patchReleaseAuditProblems.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7782/console This message was automatically generated.
          Hide
          Chris Nauroth added a comment -

          +1 for patch v009. I confirmed a full test run on Windows.

          Whitespace issues can be fixed on commit.

          I have one more very minor nitpick.

          -    if (Shell.WINDOWS && !Shell.isJava7OrAbove()) {
          -      // On Java6 on Windows, we copied the file
          -      Assert.assertEquals(data.length, link.length());
          -    } else {
          -      // Otherwise, the target file size is zero
          -      Assert.assertEquals(0, link.length());
          -    }
          +    // Otherwise, the target file size is zero
          +    Assert.assertEquals(0, link.length());
          

          Now that the conditional logic is gone, the "otherwise" phrasing of the comment doesn't make sense. I think the whole comment can be removed at this point. Feel free to drop it during the final commit. IMO, there is no need to go through full review of another patch revision for this minor tweak.

          Steve Loughran, thank you for contributing the patch.

          Show
          Chris Nauroth added a comment - +1 for patch v009. I confirmed a full test run on Windows. Whitespace issues can be fixed on commit. I have one more very minor nitpick. - if (Shell.WINDOWS && !Shell.isJava7OrAbove()) { - // On Java6 on Windows, we copied the file - Assert.assertEquals(data.length, link.length()); - } else { - // Otherwise, the target file size is zero - Assert.assertEquals(0, link.length()); - } + // Otherwise, the target file size is zero + Assert.assertEquals(0, link.length()); Now that the conditional logic is gone, the "otherwise" phrasing of the comment doesn't make sense. I think the whole comment can be removed at this point. Feel free to drop it during the final commit. IMO, there is no need to go through full review of another patch revision for this minor tweak. Steve Loughran , thank you for contributing the patch.
          Hide
          Steve Loughran added a comment -

          well spotted -I'dd do that

          Show
          Steve Loughran added a comment - well spotted -I'dd do that
          Hide
          Steve Loughran added a comment -

          thanks, committed, with your correction

          Show
          Steve Loughran added a comment - thanks, committed, with your correction
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #8625 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8625/)
          HADOOP-10775. Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #8625 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8625/ ) HADOOP-10775 . Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1259 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1259/)
          HADOOP-10775. Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1259 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1259/ ) HADOOP-10775 . Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #524 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/524/)
          HADOOP-10775. Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #524 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/524/ ) HADOOP-10775 . Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2471 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2471/)
          HADOOP-10775. Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2471 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2471/ ) HADOOP-10775 . Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #535 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/535/)
          HADOOP-10775. Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #535 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/535/ ) HADOOP-10775 . Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #491 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/491/)
          HADOOP-10775. Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #491 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/491/ ) HADOOP-10775 . Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2429 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2429/)
          HADOOP-10775. Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2429 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2429/ ) HADOOP-10775 . Shell operations to fail with meaningful errors on windows (stevel: rev c59af2fdf8f7f53117c626a895dab7fd78cf08ec) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HardLink.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerExecutor.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java

            People

            • Assignee:
              Steve Loughran
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development