Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13824

FsShell can suppress the real error if no error message is present

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1, 2.7.3
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: fs
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      The FsShell error handling assumes in displayError() that the message argument is not null. However in the case where it is this leads to a NPE which results in suppressing the actual error information since a higher level of error handling kicks in and just dumps the stack trace of the NPE instead.

      e.g.

      Exception in thread "main" java.lang.NullPointerException
      	at org.apache.hadoop.fs.FsShell.displayError(FsShell.java:304)
      	at org.apache.hadoop.fs.FsShell.run(FsShell.java:289)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
      	at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
      

      This is deeply unhelpful because depending on what the underlying error was there may be no stack dumped/logged for it (as HADOOP-7114 provides) since FsShell doesn't explicitly dump traces for IllegalArgumentException which appears to be the underlying cause of my issue. Line 289 is where displayError() is called for IllegalArgumentException handling and that catch clause does not log the error.

      1. HADOOP-13824.003.patch
        8 kB
        John Zhuge
      2. HADOOP-13824.002.patch
        10 kB
        John Zhuge
      3. HADOOP-13824.001.patch
        4 kB
        John Zhuge

        Activity

        Hide
        rvesse Rob Vesse added a comment -

        This isn't simply a case of me providing a bad argument at the command line. I am using the generic -D to pass in a valid HDFS client option.

        When I set that option to true I encounter this error, presumably because the option leads to an IllegalArgumentException somewhere in the HDFS client inner workings. But as I don't have a stack trace to go on I don't know where to start looking.

        When I set to false I avoid the NPE and the command runs (although still errors elsewhere due to network configurations which setting the option to true is a potential workaround for)

        Show
        rvesse Rob Vesse added a comment - This isn't simply a case of me providing a bad argument at the command line. I am using the generic -D to pass in a valid HDFS client option. When I set that option to true I encounter this error, presumably because the option leads to an IllegalArgumentException somewhere in the HDFS client inner workings. But as I don't have a stack trace to go on I don't know where to start looking. When I set to false I avoid the NPE and the command runs (although still errors elsewhere due to network configurations which setting the option to true is a potential workaround for)
        Hide
        jzhuge John Zhuge added a comment -

        Hi Rob Vesse, thanks a lot for reporting the issue. It will make hadoop much more usable and supportable to fix a issue like this. Could you please provide the command line for your test?

        Show
        jzhuge John Zhuge added a comment - Hi Rob Vesse , thanks a lot for reporting the issue. It will make hadoop much more usable and supportable to fix a issue like this. Could you please provide the command line for your test?
        Hide
        rvesse Rob Vesse added a comment -

        The command line is as follows:

        hdfs dfs -D dfs.client.use.datanode.hostname=true -cat hdfs://remote-namenode:8020/some/file
        

        So I am attempting to talk to a Remote HDFS cluster where I can access the name node but the data nodes are registered with internal IP addresses hence why I want to use hostnames. I think the underlying cause is that the returned hostnames are not resolvable on the machine where I run this command but since the underlying error is suppressed I can't tell for certain.

        Reproducing the specific case may be awkward because it appears to require specific network configurations. However if you look at the code for FsShell you should be able to see by inspection what I'm talking about. Specifically there is a catch branch in FsShell.run() that does not log the error and the displayError() method does not check that the provided message argument is not null

        I can probably provide a patch for this myself but I'm taking some time off this weekend so it would be Monday at the earliest

        Show
        rvesse Rob Vesse added a comment - The command line is as follows: hdfs dfs -D dfs.client.use.datanode.hostname=true -cat hdfs://remote-namenode:8020/some/file So I am attempting to talk to a Remote HDFS cluster where I can access the name node but the data nodes are registered with internal IP addresses hence why I want to use hostnames. I think the underlying cause is that the returned hostnames are not resolvable on the machine where I run this command but since the underlying error is suppressed I can't tell for certain. Reproducing the specific case may be awkward because it appears to require specific network configurations. However if you look at the code for FsShell you should be able to see by inspection what I'm talking about. Specifically there is a catch branch in FsShell.run() that does not log the error and the displayError() method does not check that the provided message argument is not null I can probably provide a patch for this myself but I'm taking some time off this weekend so it would be Monday at the earliest
        Hide
        jzhuge John Zhuge added a comment -

        Patch 001:

        • Upon catching an exception without message, print the stack trace.
        Show
        jzhuge John Zhuge added a comment - Patch 001: Upon catching an exception without message, print the stack trace.
        Hide
        jzhuge John Zhuge added a comment -

        Rob Vesse, please try the patch in your environment to catch the source of the exception without message.

        Show
        jzhuge John Zhuge added a comment - Rob Vesse , please try the patch in your environment to catch the source of the exception without message.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 40s trunk passed
        +1 compile 9m 47s trunk passed
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 1m 5s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 1m 28s trunk passed
        +1 javadoc 0m 48s trunk passed
        +1 mvninstall 0m 39s the patch passed
        +1 compile 9m 23s the patch passed
        +1 javac 9m 23s the patch passed
        +1 checkstyle 0m 29s the patch passed
        +1 mvnsite 1m 0s the patch passed
        +1 mvneclipse 0m 19s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 33s the patch passed
        +1 javadoc 0m 47s the patch passed
        +1 unit 8m 23s hadoop-common in the patch passed.
        +1 asflicense 0m 31s The patch does not generate ASF License warnings.
        45m 40s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HADOOP-13824
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839532/HADOOP-13824.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 342f837608bf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c0b1a44
        Default Java 1.8.0_111
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11097/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11097/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 40s trunk passed +1 compile 9m 47s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 5s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 1m 28s trunk passed +1 javadoc 0m 48s trunk passed +1 mvninstall 0m 39s the patch passed +1 compile 9m 23s the patch passed +1 javac 9m 23s the patch passed +1 checkstyle 0m 29s the patch passed +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 33s the patch passed +1 javadoc 0m 47s the patch passed +1 unit 8m 23s hadoop-common in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 45m 40s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13824 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839532/HADOOP-13824.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 342f837608bf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c0b1a44 Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11097/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11097/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        patchs looks good, except I'd go

         displayError(cmd, ex.toString());
        
        Show
        stevel@apache.org Steve Loughran added a comment - patchs looks good, except I'd go displayError(cmd, ex.toString());
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        John Zhuge thanks for the patch. You probably don't need this line in the test:

        Mockito.when(mockCommand.getDescription()).thenReturn("A mock command");
        

        Other than that, the patch looks good to me.

        Show
        jojochuang Wei-Chiu Chuang added a comment - John Zhuge thanks for the patch. You probably don't need this line in the test: Mockito.when(mockCommand.getDescription()).thenReturn( "A mock command" ); Other than that, the patch looks good to me.
        Hide
        rvesse Rob Vesse added a comment -

        Yes this patch worked for me, I was able to see the underlying error which was the following:

        16/11/21 06:16:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
        -cat: Caught exception without message
        java.nio.channels.UnresolvedAddressException
        	at sun.nio.ch.Net.checkAddress(Net.java:157)
        	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:703)
        	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
        	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
        	at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3436)
        	at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:777)
        	at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:694)
        	at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:355)
        	at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:656)
        	at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:882)
        	at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:934)
        	at java.io.DataInputStream.read(DataInputStream.java:111)
        	at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:85)
        	at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:59)
        	at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:119)
        	at org.apache.hadoop.fs.shell.Display$Cat.printToStdout(Display.java:107)
        	at org.apache.hadoop.fs.shell.Display$Cat.processPath(Display.java:102)
        	at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:317)
        	at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:289)
        	at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:271)
        	at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:255)
        	at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:201)
        	at org.apache.hadoop.fs.shell.Command.run(Command.java:165)
        	at org.apache.hadoop.fs.FsShell.run(FsShell.java:287)
        	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        	at org.apache.hadoop.fs.FsShell.main(FsShell.java:345)
        Usage: hadoop fs [generic options] -cat [-ignoreCrc] <src> ...
        

        This was the actual error I was expecting to see

        Show
        rvesse Rob Vesse added a comment - Yes this patch worked for me, I was able to see the underlying error which was the following: 16/11/21 06:16:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable -cat: Caught exception without message java.nio.channels.UnresolvedAddressException at sun.nio.ch.Net.checkAddress(Net.java:157) at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:703) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3436) at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:777) at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:694) at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:355) at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:656) at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:882) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:934) at java.io.DataInputStream.read(DataInputStream.java:111) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:85) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:59) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:119) at org.apache.hadoop.fs.shell.Display$Cat.printToStdout(Display.java:107) at org.apache.hadoop.fs.shell.Display$Cat.processPath(Display.java:102) at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:317) at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:289) at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:271) at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:255) at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:201) at org.apache.hadoop.fs.shell.Command.run(Command.java:165) at org.apache.hadoop.fs.FsShell.run(FsShell.java:287) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hadoop.fs.FsShell.main(FsShell.java:345) Usage: hadoop fs [generic options] -cat [-ignoreCrc] <src> ... This was the actual error I was expecting to see
        Hide
        rvesse Rob Vesse added a comment -

        Btw what branch was this patch prepared against? I had to apply it manually to the 2.7.3 release tag since we are using 2.7.x in our environment

        Show
        rvesse Rob Vesse added a comment - Btw what branch was this patch prepared against? I had to apply it manually to the 2.7.3 release tag since we are using 2.7.x in our environment
        Hide
        stevel@apache.org Steve Loughran added a comment -

        any patch without a branchname is against trunk; if something says "HADOOP-12824-branch-2-001.path" yetus would know to apply to branch 2. To apply to branch-2.7, that'd need to be the branch name in the patch (and yes, the hadoop branch you should be applying to locally)

        Show
        stevel@apache.org Steve Loughran added a comment - any patch without a branchname is against trunk; if something says " HADOOP-12824 -branch-2-001.path" yetus would know to apply to branch 2. To apply to branch-2.7, that'd need to be the branch name in the patch (and yes, the hadoop branch you should be applying to locally)
        Hide
        jzhuge John Zhuge added a comment - - edited

        Patch 002:

        • Add test utilility class SystemErrCapture and TeePrintStream
        • Wei-Chiu's suggestion

        Steve Loughran, can not use ex.toString() because it would break testDFSWithInvalidCommmand.

        Show
        jzhuge John Zhuge added a comment - - edited Patch 002: Add test utilility class SystemErrCapture and TeePrintStream Wei-Chiu's suggestion Steve Loughran , can not use ex.toString() because it would break testDFSWithInvalidCommmand .
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        +1 mvninstall 9m 13s trunk passed
        +1 compile 12m 9s trunk passed
        +1 checkstyle 0m 32s trunk passed
        +1 mvnsite 1m 17s trunk passed
        +1 mvneclipse 0m 21s trunk passed
        +1 findbugs 1m 37s trunk passed
        +1 javadoc 0m 52s trunk passed
        +1 mvninstall 0m 48s the patch passed
        +1 compile 11m 25s the patch passed
        +1 javac 11m 25s the patch passed
        +1 checkstyle 0m 33s hadoop-common-project/hadoop-common: The patch generated 0 new + 19 unchanged - 1 fixed = 19 total (was 20)
        +1 mvnsite 1m 14s the patch passed
        +1 mvneclipse 0m 21s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 0s the patch passed
        +1 javadoc 0m 56s the patch passed
        +1 unit 8m 53s hadoop-common in the patch passed.
        +1 asflicense 0m 30s The patch does not generate ASF License warnings.
        54m 46s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HADOOP-13824
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840387/HADOOP-13824.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 510af050043e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e15c20e
        Default Java 1.8.0_111
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11128/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11128/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 9m 13s trunk passed +1 compile 12m 9s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 17s trunk passed +1 mvneclipse 0m 21s trunk passed +1 findbugs 1m 37s trunk passed +1 javadoc 0m 52s trunk passed +1 mvninstall 0m 48s the patch passed +1 compile 11m 25s the patch passed +1 javac 11m 25s the patch passed +1 checkstyle 0m 33s hadoop-common-project/hadoop-common: The patch generated 0 new + 19 unchanged - 1 fixed = 19 total (was 20) +1 mvnsite 1m 14s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 0s the patch passed +1 javadoc 0m 56s the patch passed +1 unit 8m 53s hadoop-common in the patch passed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 54m 46s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13824 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840387/HADOOP-13824.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 510af050043e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e15c20e Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11128/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11128/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        John Zhuge thanks for the patch. I reviewed it and have just spotted two very tiny nits. Both are subjective to my personal preference:

        • StringContains#containsString may be replaced by {GenericTestUtils#assertMatches.
        • I feel SystemErrCapture should go into GenericTestUtils. There's a LogCapturer class in GenericTestUtils serving the similar purpose for logs. But that's purely my personal preference.
        Show
        jojochuang Wei-Chiu Chuang added a comment - John Zhuge thanks for the patch. I reviewed it and have just spotted two very tiny nits. Both are subjective to my personal preference: StringContains#containsString may be replaced by { GenericTestUtils#assertMatches . I feel SystemErrCapture should go into GenericTestUtils . There's a LogCapturer class in GenericTestUtils serving the similar purpose for logs. But that's purely my personal preference.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Forgot to mention I am +1 for the patch and will commit soon.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Forgot to mention I am +1 for the patch and will commit soon.
        Hide
        jzhuge John Zhuge added a comment -

        Patch 003:

        • Wei-Chiu's comment

        GenericTestUtils#assertMatches does regex not substring match. assertThat + containStrings also prints expected and actual strings.

        Show
        jzhuge John Zhuge added a comment - Patch 003: Wei-Chiu's comment GenericTestUtils#assertMatches does regex not substring match. assertThat + containStrings also prints expected and actual strings.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 1s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 30s trunk passed
        +1 compile 11m 2s trunk passed
        +1 checkstyle 0m 33s trunk passed
        +1 mvnsite 1m 12s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 1m 40s trunk passed
        +1 javadoc 0m 57s trunk passed
        +1 mvninstall 0m 46s the patch passed
        -1 compile 10m 21s root in the patch failed.
        -1 javac 10m 21s root in the patch failed.
        +1 checkstyle 0m 32s hadoop-common-project/hadoop-common: The patch generated 0 new + 35 unchanged - 1 fixed = 35 total (was 36)
        +1 mvnsite 1m 10s the patch passed
        +1 mvneclipse 0m 16s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 40s the patch passed
        +1 javadoc 0m 51s the patch passed
        -1 unit 9m 8s hadoop-common in the patch failed.
        +1 asflicense 0m 35s The patch does not generate ASF License warnings.
        51m 30s



        Reason Tests
        Failed junit tests hadoop.test.TestLambdaTestUtils



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HADOOP-13824
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842418/HADOOP-13824.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux a98df8f8c038 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 13d8e55
        Default Java 1.8.0_111
        findbugs v3.0.0
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/artifact/patchprocess/patch-compile-root.txt
        javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/artifact/patchprocess/patch-compile-root.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 1s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 30s trunk passed +1 compile 11m 2s trunk passed +1 checkstyle 0m 33s trunk passed +1 mvnsite 1m 12s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 1m 40s trunk passed +1 javadoc 0m 57s trunk passed +1 mvninstall 0m 46s the patch passed -1 compile 10m 21s root in the patch failed. -1 javac 10m 21s root in the patch failed. +1 checkstyle 0m 32s hadoop-common-project/hadoop-common: The patch generated 0 new + 35 unchanged - 1 fixed = 35 total (was 36) +1 mvnsite 1m 10s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 40s the patch passed +1 javadoc 0m 51s the patch passed -1 unit 9m 8s hadoop-common in the patch failed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 51m 30s Reason Tests Failed junit tests hadoop.test.TestLambdaTestUtils Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13824 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842418/HADOOP-13824.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a98df8f8c038 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 13d8e55 Default Java 1.8.0_111 findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/artifact/patchprocess/patch-compile-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11222/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jzhuge John Zhuge added a comment -

        compile and javac had the following error probably due to intermittent github connection issue. The same link works now.

        bower datatables#~1.10.8                   ECMDERR Failed to execute "git ls-remote --tags --heads https://github.com/DataTables/DataTables.git", exit code of #128 fatal: unable to access 'https://github.com/DataTables/DataTables.git/': Failed to connect to github.com port 443: Connection timed out
        

        TestLambdaTestUtils failure is unrelated.

        Show
        jzhuge John Zhuge added a comment - compile and javac had the following error probably due to intermittent github connection issue. The same link works now. bower datatables#~1.10.8 ECMDERR Failed to execute "git ls-remote --tags --heads https://github.com/DataTables/DataTables.git", exit code of #128 fatal: unable to access 'https://github.com/DataTables/DataTables.git/': Failed to connect to github.com port 443: Connection timed out TestLambdaTestUtils failure is unrelated.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        LGTM

        +1

        Filed a JIRA on the LTU test failure

        Show
        stevel@apache.org Steve Loughran added a comment - LGTM +1 Filed a JIRA on the LTU test failure
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Committing 003 patch.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Committing 003 patch.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10978 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10978/)
        HADOOP-13824. FsShell can suppress the real error if no error message is (weichiu: rev b606e025f10daed18b90b45ac00cd0c82e818581)

        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShell.java
        • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShell.java
        • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10978 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10978/ ) HADOOP-13824 . FsShell can suppress the real error if no error message is (weichiu: rev b606e025f10daed18b90b45ac00cd0c82e818581) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsShell.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShell.java (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Committed to trunk, branch-2 and branch-2.8.

        Thanks to Rob Vesse for reporting the issue, John Zhuge for contributing the patch and Steve Loughran for reviewing it!

        Show
        jojochuang Wei-Chiu Chuang added a comment - Committed to trunk, branch-2 and branch-2.8. Thanks to Rob Vesse for reporting the issue, John Zhuge for contributing the patch and Steve Loughran for reviewing it!
        Hide
        jzhuge John Zhuge added a comment -

        Thanks Wei-Chiu Chuang for the review and commit! Thanks Rob Vesse for reporting and Steve Loughran for the review.

        Show
        jzhuge John Zhuge added a comment - Thanks Wei-Chiu Chuang for the review and commit! Thanks Rob Vesse for reporting and Steve Loughran for the review.

          People

          • Assignee:
            jzhuge John Zhuge
            Reporter:
            rvesse Rob Vesse
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development