Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10276

HDFS should not expose path info that user has no permission to see.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This following issue is remedied by HDFS-5802.

      Given you have a file /file an existence check for the path /file/whatever will give different responses for different implementations of FileSystem.

      LocalFileSystem will return false while DistributedFileSystem will throw org.apache.hadoop.security.AccessControlException: Permission denied: ..., access=EXECUTE, ...

      However, HDFS-5802 may expose information about a path that user doesn't have permission to see.

      For example, if the user asks for /a/b/c, but does not have permission to list /a, we should not complain about /a/b

      1. HDFS-10276.001.patch
        2 kB
        Yuanbo Liu
      2. HDFS-10276.002.patch
        3 kB
        Yuanbo Liu
      3. HDFS-10276.003.patch
        12 kB
        Yuanbo Liu
      4. HDFS-10276.004.patch
        11 kB
        Yuanbo Liu
      5. HDFS-10276.005.patch
        3 kB
        Yuanbo Liu
      6. HDFS-10276.006.patch
        4 kB
        Yuanbo Liu

        Issue Links

          Activity

          Hide
          yuanbo Yuanbo Liu added a comment -

          Hi, Kevin
          Can you provide more details about this issue? such as the steps of reproduction, the version of Hadoop.
          I have tested LocalFileSystem/DistributedFileSystem#getFileStatus and LocalFileSystem/DistributedFileSystem#exists on Hadoop trunk and they both worked as expected.

          Show
          yuanbo Yuanbo Liu added a comment - Hi, Kevin Can you provide more details about this issue? such as the steps of reproduction, the version of Hadoop. I have tested LocalFileSystem/DistributedFileSystem#getFileStatus and LocalFileSystem/DistributedFileSystem#exists on Hadoop trunk and they both worked as expected.
          Hide
          kevincox Kevin Cox added a comment -

          I have tested on two moderately recent versions of hadoop and the following commands will reproduce the issue. The chmod is unnecessary in most cases but I have included it just to be sure. The following outputs are on a HDFS. When run on a local path

          % hdfs version
          Hadoop 2.7.2
          Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b165c4fe8a74265c792ce23f546c64604acf0e41
          Compiled by jenkins on 2016-01-26T00:08Z
          Compiled with protoc 2.5.0
          From source with checksum d0fda26633fa762bff87ec759ebe689c
          This command was run using /usr/local/Cellar/hadoop/2.7.2/libexec/share/hadoop/common/hadoop-common-2.7.2.jar
          % hdfs --config hadoop dfs -put <(echo "test") /test
          2016-04-13 09:24:58,851 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          hdfs --config hadoop dfs -put <(echo "test") /test  2.74s user 0.21s system 180% cpu 1.638 total
          % hdfs --config hadoop dfs -chmod 777 /test         
          2016-04-13 09:25:11,737 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          % hdfs --config hadoop dfs -cat /test/bar           
          2016-04-13 09:25:18,905 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          cat: `/test/bar': No such file or directory
          % hdfs --config hadoop dfs -cat file:///home/kevincox/test/foo
          2016-04-13 09:29:34,150 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          cat: `file:///Users/kevincox/test/foo': No such file or directory
          

          This isn't a huge issue for the command line client because it returns the same exit code but in particular for the FileSystem.exists() method this is the difference between returning false and throwing an error.

          Show
          kevincox Kevin Cox added a comment - I have tested on two moderately recent versions of hadoop and the following commands will reproduce the issue. The chmod is unnecessary in most cases but I have included it just to be sure. The following outputs are on a HDFS. When run on a local path % hdfs version Hadoop 2.7.2 Subversion https: //git-wip-us.apache.org/repos/asf/hadoop.git -r b165c4fe8a74265c792ce23f546c64604acf0e41 Compiled by jenkins on 2016-01-26T00:08Z Compiled with protoc 2.5.0 From source with checksum d0fda26633fa762bff87ec759ebe689c This command was run using /usr/local/Cellar/hadoop/2.7.2/libexec/share/hadoop/common/hadoop-common-2.7.2.jar % hdfs --config hadoop dfs -put <(echo "test" ) /test 2016-04-13 09:24:58,851 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native -hadoop library for your platform... using builtin-java classes where applicable hdfs --config hadoop dfs -put <(echo "test" ) /test 2.74s user 0.21s system 180% cpu 1.638 total % hdfs --config hadoop dfs -chmod 777 /test 2016-04-13 09:25:11,737 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native -hadoop library for your platform... using builtin-java classes where applicable % hdfs --config hadoop dfs -cat /test/bar 2016-04-13 09:25:18,905 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native -hadoop library for your platform... using builtin-java classes where applicable cat: `/test/bar': No such file or directory % hdfs --config hadoop dfs -cat file: ///home/kevincox/test/foo 2016-04-13 09:29:34,150 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native -hadoop library for your platform... using builtin-java classes where applicable cat: `file: ///Users/kevincox/test/foo': No such file or directory This isn't a huge issue for the command line client because it returns the same exit code but in particular for the FileSystem.exists() method this is the difference between returning false and throwing an error.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Hi, Kevin
          Sorry for taking so long to reply you, I'm quite busy last week.
          Seems that you used the username kevincox to operate files of root directory on the distributed file system. I think the distributed file system works as expected since you may not have the access to operate the files. It's better to compare the two different file systems under the same condition, for instance, the username kevincox does not have the access to read/write/execute files on the both systems and then operate the files. I've tested different scenarios and exists method did not throw any error. So could you please provide the steps of throwing org.apache.hadoop.security.AccessControlException: Permission denied: ..., access=EXECUTE, ... ?

          Show
          yuanbo Yuanbo Liu added a comment - Hi, Kevin Sorry for taking so long to reply you, I'm quite busy last week. Seems that you used the username kevincox to operate files of root directory on the distributed file system. I think the distributed file system works as expected since you may not have the access to operate the files. It's better to compare the two different file systems under the same condition, for instance, the username kevincox does not have the access to read/write/execute files on the both systems and then operate the files. I've tested different scenarios and exists method did not throw any error. So could you please provide the steps of throwing org.apache.hadoop.security.AccessControlException: Permission denied: ..., access=EXECUTE, ... ?
          Hide
          yuanbo Yuanbo Liu added a comment -

          Hi, Kevin
          Sorry for taking so long to reply you, I'm quite busy last week.
          Seems that you used the username kevincox to operate files of root directory on the distributed file system. I think the distributed file system works as expected since you may not have the access to operate the files. It's better to compare the two different file systems under the same condition, for instance, the username kevincox does not have the access to read/write/execute files on the both systems and then operate the files. I've tested different scenarios and exists method did not throw any error. So could you please provide the steps of throwing org.apache.hadoop.security.AccessControlException: Permission denied: ..., access=EXECUTE, ... ?

          Show
          yuanbo Yuanbo Liu added a comment - Hi, Kevin Sorry for taking so long to reply you, I'm quite busy last week. Seems that you used the username kevincox to operate files of root directory on the distributed file system. I think the distributed file system works as expected since you may not have the access to operate the files. It's better to compare the two different file systems under the same condition, for instance, the username kevincox does not have the access to read/write/execute files on the both systems and then operate the files. I've tested different scenarios and exists method did not throw any error. So could you please provide the steps of throwing org.apache.hadoop.security.AccessControlException: Permission denied: ..., access=EXECUTE, ... ?
          Hide
          kevincox Kevin Cox added a comment -

          Oops, my mistake. That chmod should be 666 (non executable) which would cause the `cat` to throw the exception. The new log is below.

          % hdfs --config starscream/hadoop dfs -put <(echo test) /test
          2016-04-18 08:15:41,615 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          hdfs --config starscream/hadoop dfs -put <(echo test) /test  2.72s user 0.18s system 128% cpu 2.269 total
          % hdfs --config starscream/hadoop dfs -chmod 666 /test                   
          2016-04-18 08:16:52,903 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          hdfs --config starscream/hadoop dfs -chmod 666 /test  2.37s user 0.16s system 182% cpu 1.390 total
          % hdfs --config starscream/hadoop dfs -cat /test/bar
          2016-04-18 08:16:55,743 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          cat: Permission denied: user=foo, access=EXECUTE, inode="/test/bar":kevincox:supergroup:-rw-rw-rw-
          HADOOP_USER_NAME=foo hdfs --config starscream/hadoop dfs -cat /test/bar  2.40s user 0.16s system 185% cpu 1.378 total
          
          Show
          kevincox Kevin Cox added a comment - Oops, my mistake. That chmod should be 666 (non executable) which would cause the `cat` to throw the exception. The new log is below. % hdfs --config starscream/hadoop dfs -put <(echo test) /test 2016-04-18 08:15:41,615 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native -hadoop library for your platform... using builtin-java classes where applicable hdfs --config starscream/hadoop dfs -put <(echo test) /test 2.72s user 0.18s system 128% cpu 2.269 total % hdfs --config starscream/hadoop dfs -chmod 666 /test 2016-04-18 08:16:52,903 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native -hadoop library for your platform... using builtin-java classes where applicable hdfs --config starscream/hadoop dfs -chmod 666 /test 2.37s user 0.16s system 182% cpu 1.390 total % hdfs --config starscream/hadoop dfs -cat /test/bar 2016-04-18 08:16:55,743 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native -hadoop library for your platform... using builtin-java classes where applicable cat: Permission denied: user=foo, access=EXECUTE, inode= "/test/bar" :kevincox:supergroup:-rw-rw-rw- HADOOP_USER_NAME=foo hdfs --config starscream/hadoop dfs -cat /test/bar 2.40s user 0.16s system 185% cpu 1.378 total
          Hide
          yuanbo Yuanbo Liu added a comment -

          Kevin CoxThanks a lot for your comment. I've reproduced this error and it should be marked as a bug. I found that name node would check the access before checking whether the file existed. It takes time to fix this bug since I have to find out whether other file systems work as expected and how to fix it appositely.

          Show
          yuanbo Yuanbo Liu added a comment - Kevin Cox Thanks a lot for your comment. I've reproduced this error and it should be marked as a bug. I found that name node would check the access before checking whether the file existed. It takes time to fix this bug since I have to find out whether other file systems work as expected and how to fix it appositely.
          Hide
          kevincox Kevin Cox added a comment -

          Excellent. As I said we originally noticed this behaviour because of the difference between behaviour on HDFS and local. I haven't tested with any other implementations.

          Show
          kevincox Kevin Cox added a comment - Excellent. As I said we originally noticed this behaviour because of the difference between behaviour on HDFS and local. I haven't tested with any other implementations.
          Hide
          cmccabe Colin P. McCabe added a comment - - edited

          Thanks a lot for your comment. I've reproduced this error and it should be marked as a bug. I found that name node would check the access before checking whether the file existed.

          The NameNode needs to check access before checking whether the file exists. Otherwise, unprivileged users could get information about files and directories they should not have access to. In this specific case, though, we do want exists to return false, since the user apparently does have permissions to find out that the path doesn't exist.

          Show
          cmccabe Colin P. McCabe added a comment - - edited Thanks a lot for your comment. I've reproduced this error and it should be marked as a bug. I found that name node would check the access before checking whether the file existed. The NameNode needs to check access before checking whether the file exists. Otherwise, unprivileged users could get information about files and directories they should not have access to. In this specific case, though, we do want exists to return false, since the user apparently does have permissions to find out that the path doesn't exist.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Colin P. McCabe Thanks for your comment. Yes I agree with you that we should take unprivileged users as consider. But in this case, NameNode should prevent any further operation since it knows the "parent path" is a file, not a real path. I think it's more reasonable to throw ParentNotDirectoryException rather than AccessControlException. See the behavior of Linux as below:

          [yuanbo@oc7702007844 ~]$ ls test/test1
          ls: cannot access test/test1: Not a directory
          

          One thing should be rectified is that getFileInfo in DFSClient causes this problem instead of exists method.

          Show
          yuanbo Yuanbo Liu added a comment - Colin P. McCabe Thanks for your comment. Yes I agree with you that we should take unprivileged users as consider. But in this case, NameNode should prevent any further operation since it knows the "parent path" is a file, not a real path. I think it's more reasonable to throw ParentNotDirectoryException rather than AccessControlException . See the behavior of Linux as below: [yuanbo@oc7702007844 ~]$ ls test/test1 ls: cannot access test/test1: Not a directory One thing should be rectified is that getFileInfo in DFSClient causes this problem instead of exists method.
          Hide
          cmccabe Colin P. McCabe added a comment -

          I think it's more reasonable to throw ParentNotDirectoryException rather than AccessControlException.

          I agree. This seems like it would be an incompatible change, though, so probably branch-3 only.

          Show
          cmccabe Colin P. McCabe added a comment - I think it's more reasonable to throw ParentNotDirectoryException rather than AccessControlException. I agree. This seems like it would be an incompatible change, though, so probably branch-3 only.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 29s Maven dependency ordering for branch
          +1 mvninstall 9m 49s trunk passed
          +1 compile 2m 24s trunk passed with JDK v1.8.0_77
          +1 compile 2m 0s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 34s trunk passed
          +1 mvnsite 2m 16s trunk passed
          +1 mvneclipse 0m 37s trunk passed
          +1 findbugs 5m 1s trunk passed
          +1 javadoc 2m 23s trunk passed with JDK v1.8.0_77
          +1 javadoc 3m 15s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 13s Maven dependency ordering for patch
          +1 mvninstall 2m 1s the patch passed
          +1 compile 2m 34s the patch passed with JDK v1.8.0_77
          +1 javac 2m 34s the patch passed
          +1 compile 2m 1s the patch passed with JDK v1.7.0_95
          +1 javac 2m 1s the patch passed
          +1 checkstyle 0m 34s the patch passed
          +1 mvnsite 1m 53s the patch passed
          +1 mvneclipse 0m 32s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 5m 24s the patch passed
          +1 javadoc 2m 26s the patch passed with JDK v1.8.0_77
          +1 javadoc 2m 59s the patch passed with JDK v1.7.0_95
          +1 unit 1m 30s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77.
          -1 unit 110m 28s hadoop-hdfs in the patch failed with JDK v1.8.0_77.
          +1 unit 1m 46s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 94m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 24s Patch does not generate ASF License warnings.
          262m 24s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID
            hadoop.hdfs.TestDFSPermission
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
            hadoop.fs.TestWebHdfsFileContextMainOperations
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.server.blockmanagement.TestBlockManager
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.TestHDFSFileSystemContract
            hadoop.hdfs.server.namenode.TestINodeFile
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.fs.TestSWebHdfsFileContextMainOperations
            hadoop.hdfs.TestReservedRawPaths
            hadoop.cli.TestXAttrCLI
            hadoop.hdfs.TestDFSMkdirs
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.fs.TestHDFSFileContextMainOperations
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.web.TestWebHdfsFileSystemContract
            hadoop.hdfs.TestDFSShell
            hadoop.fs.TestSymlinkHdfsFileContext
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.qjournal.TestSecureNNWithQJM
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.web.TestFSMainOperationsWebHdfs
            hadoop.hdfs.TestEncryptionZonesWithKMS
            hadoop.fs.TestGlobPaths
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestDFSPermission
            hadoop.fs.TestWebHdfsFileContextMainOperations
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
            hadoop.hdfs.TestHDFSFileSystemContract
            hadoop.hdfs.server.namenode.TestINodeFile
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.fs.TestSWebHdfsFileContextMainOperations
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.TestReservedRawPaths
            hadoop.cli.TestXAttrCLI
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.TestDFSMkdirs
            hadoop.fs.TestHDFSFileContextMainOperations
            hadoop.hdfs.web.TestWebHdfsFileSystemContract
            hadoop.hdfs.TestDFSShell
            hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.web.TestFSMainOperationsWebHdfs
            hadoop.hdfs.TestEncryptionZonesWithKMS
            hadoop.fs.TestGlobPaths
            hadoop.hdfs.server.datanode.TestDirectoryScanner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800474/HDFS-10276.002.patch
          JIRA Issue HDFS-10276
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 0ab1008c8139 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c282a08
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15270/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15270/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15270/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15270/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15270/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15270/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 29s Maven dependency ordering for branch +1 mvninstall 9m 49s trunk passed +1 compile 2m 24s trunk passed with JDK v1.8.0_77 +1 compile 2m 0s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 34s trunk passed +1 mvnsite 2m 16s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 5m 1s trunk passed +1 javadoc 2m 23s trunk passed with JDK v1.8.0_77 +1 javadoc 3m 15s trunk passed with JDK v1.7.0_95 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 2m 1s the patch passed +1 compile 2m 34s the patch passed with JDK v1.8.0_77 +1 javac 2m 34s the patch passed +1 compile 2m 1s the patch passed with JDK v1.7.0_95 +1 javac 2m 1s the patch passed +1 checkstyle 0m 34s the patch passed +1 mvnsite 1m 53s the patch passed +1 mvneclipse 0m 32s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 24s the patch passed +1 javadoc 2m 26s the patch passed with JDK v1.8.0_77 +1 javadoc 2m 59s the patch passed with JDK v1.7.0_95 +1 unit 1m 30s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. -1 unit 110m 28s hadoop-hdfs in the patch failed with JDK v1.8.0_77. +1 unit 1m 46s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 94m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 262m 24s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.TestDFSPermission   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.fs.TestWebHdfsFileContextMainOperations   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.TestHDFSFileSystemContract   hadoop.hdfs.server.namenode.TestINodeFile   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.fs.TestSWebHdfsFileContextMainOperations   hadoop.hdfs.TestReservedRawPaths   hadoop.cli.TestXAttrCLI   hadoop.hdfs.TestDFSMkdirs   hadoop.hdfs.server.namenode.TestEditLog   hadoop.fs.TestHDFSFileContextMainOperations   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.web.TestWebHdfsFileSystemContract   hadoop.hdfs.TestDFSShell   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.web.TestFSMainOperationsWebHdfs   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.fs.TestGlobPaths   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestDFSPermission   hadoop.fs.TestWebHdfsFileContextMainOperations   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   hadoop.hdfs.TestHDFSFileSystemContract   hadoop.hdfs.server.namenode.TestINodeFile   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.fs.TestSWebHdfsFileContextMainOperations   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestReservedRawPaths   hadoop.cli.TestXAttrCLI   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.TestDFSMkdirs   hadoop.fs.TestHDFSFileContextMainOperations   hadoop.hdfs.web.TestWebHdfsFileSystemContract   hadoop.hdfs.TestDFSShell   hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.web.TestFSMainOperationsWebHdfs   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.fs.TestGlobPaths   hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800474/HDFS-10276.002.patch JIRA Issue HDFS-10276 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0ab1008c8139 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c282a08 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15270/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15270/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15270/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15270/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15270/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15270/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          0 mvndep 0m 30s Maven dependency ordering for branch
          +1 mvninstall 7m 8s trunk passed
          +1 compile 6m 21s trunk passed with JDK v1.8.0_91
          +1 compile 6m 55s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 6s trunk passed
          +1 mvnsite 2m 24s trunk passed
          +1 mvneclipse 0m 38s trunk passed
          +1 findbugs 5m 27s trunk passed
          +1 javadoc 2m 26s trunk passed with JDK v1.8.0_91
          +1 javadoc 3m 20s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 2m 2s the patch passed
          +1 compile 6m 13s the patch passed with JDK v1.8.0_91
          +1 javac 6m 13s the patch passed
          +1 compile 6m 59s the patch passed with JDK v1.7.0_95
          +1 javac 6m 59s the patch passed
          -1 checkstyle 1m 8s root: patch generated 2 new + 114 unchanged - 0 fixed = 116 total (was 114)
          +1 mvnsite 2m 29s the patch passed
          +1 mvneclipse 0m 41s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 6m 15s the patch passed
          +1 javadoc 2m 36s the patch passed with JDK v1.8.0_91
          +1 javadoc 3m 33s the patch passed with JDK v1.7.0_95
          -1 unit 19m 37s hadoop-common in the patch failed with JDK v1.8.0_91.
          +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.8.0_91.
          -1 unit 64m 27s hadoop-hdfs in the patch failed with JDK v1.8.0_91.
          -1 unit 7m 47s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 1m 3s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 55m 50s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 30s Patch does not generate ASF License warnings.
          220m 34s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestFsDatasetCache
            hadoop.hdfs.TestDFSShell
            hadoop.hdfs.TestHFlush
            hadoop.hdfs.TestFileAppend
          JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle
          JDK v1.7.0_95 Failed junit tests hadoop.net.TestClusterTopology
            hadoop.security.ssl.TestReloadingX509TrustManager
            hadoop.hdfs.TestDFSShell
            hadoop.hdfs.TestHFlush



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:7b1c37a
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801201/HDFS-10276.003.patch
          JIRA Issue HDFS-10276
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7435f27e42f4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6f26b66
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15315/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15315/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. 0 mvndep 0m 30s Maven dependency ordering for branch +1 mvninstall 7m 8s trunk passed +1 compile 6m 21s trunk passed with JDK v1.8.0_91 +1 compile 6m 55s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 6s trunk passed +1 mvnsite 2m 24s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 5m 27s trunk passed +1 javadoc 2m 26s trunk passed with JDK v1.8.0_91 +1 javadoc 3m 20s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 2s the patch passed +1 compile 6m 13s the patch passed with JDK v1.8.0_91 +1 javac 6m 13s the patch passed +1 compile 6m 59s the patch passed with JDK v1.7.0_95 +1 javac 6m 59s the patch passed -1 checkstyle 1m 8s root: patch generated 2 new + 114 unchanged - 0 fixed = 116 total (was 114) +1 mvnsite 2m 29s the patch passed +1 mvneclipse 0m 41s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 6m 15s the patch passed +1 javadoc 2m 36s the patch passed with JDK v1.8.0_91 +1 javadoc 3m 33s the patch passed with JDK v1.7.0_95 -1 unit 19m 37s hadoop-common in the patch failed with JDK v1.8.0_91. +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.8.0_91. -1 unit 64m 27s hadoop-hdfs in the patch failed with JDK v1.8.0_91. -1 unit 7m 47s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 1m 3s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 55m 50s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 30s Patch does not generate ASF License warnings. 220m 34s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestFsDatasetCache   hadoop.hdfs.TestDFSShell   hadoop.hdfs.TestHFlush   hadoop.hdfs.TestFileAppend JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle JDK v1.7.0_95 Failed junit tests hadoop.net.TestClusterTopology   hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.hdfs.TestDFSShell   hadoop.hdfs.TestHFlush Subsystem Report/Notes Docker Image:yetus/hadoop:7b1c37a JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801201/HDFS-10276.003.patch JIRA Issue HDFS-10276 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7435f27e42f4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6f26b66 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15315/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15315/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15315/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          0 mvndep 0m 32s Maven dependency ordering for branch
          +1 mvninstall 6m 46s trunk passed
          +1 compile 6m 43s trunk passed with JDK v1.8.0_91
          +1 compile 7m 7s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 5s trunk passed
          +1 mvnsite 2m 25s trunk passed
          +1 mvneclipse 0m 41s trunk passed
          +1 findbugs 5m 12s trunk passed
          +1 javadoc 2m 24s trunk passed with JDK v1.8.0_91
          +1 javadoc 3m 19s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 2m 1s the patch passed
          +1 compile 6m 35s the patch passed with JDK v1.8.0_91
          +1 javac 6m 35s the patch passed
          +1 compile 7m 11s the patch passed with JDK v1.7.0_95
          +1 javac 7m 11s the patch passed
          -1 checkstyle 1m 5s root: patch generated 1 new + 114 unchanged - 0 fixed = 115 total (was 114)
          +1 mvnsite 2m 24s the patch passed
          +1 mvneclipse 0m 42s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 5m 58s the patch passed
          +1 javadoc 2m 23s the patch passed with JDK v1.8.0_91
          +1 javadoc 3m 18s the patch passed with JDK v1.7.0_95
          +1 unit 7m 53s hadoop-common in the patch passed with JDK v1.8.0_91.
          +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.8.0_91.
          -1 unit 63m 36s hadoop-hdfs in the patch failed with JDK v1.8.0_91.
          -1 unit 7m 42s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 57m 6s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 27s Patch does not generate ASF License warnings.
          208m 37s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.hdfs.TestFileAppend
          JDK v1.7.0_95 Failed junit tests hadoop.ipc.TestRPCWaitForProxy
            hadoop.hdfs.TestHFlush



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801405/HDFS-10276.004.patch
          JIRA Issue HDFS-10276
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6b1b55488e29 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4a85085
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15323/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15323/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. 0 mvndep 0m 32s Maven dependency ordering for branch +1 mvninstall 6m 46s trunk passed +1 compile 6m 43s trunk passed with JDK v1.8.0_91 +1 compile 7m 7s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 5s trunk passed +1 mvnsite 2m 25s trunk passed +1 mvneclipse 0m 41s trunk passed +1 findbugs 5m 12s trunk passed +1 javadoc 2m 24s trunk passed with JDK v1.8.0_91 +1 javadoc 3m 19s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 1s the patch passed +1 compile 6m 35s the patch passed with JDK v1.8.0_91 +1 javac 6m 35s the patch passed +1 compile 7m 11s the patch passed with JDK v1.7.0_95 +1 javac 7m 11s the patch passed -1 checkstyle 1m 5s root: patch generated 1 new + 114 unchanged - 0 fixed = 115 total (was 114) +1 mvnsite 2m 24s the patch passed +1 mvneclipse 0m 42s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 58s the patch passed +1 javadoc 2m 23s the patch passed with JDK v1.8.0_91 +1 javadoc 3m 18s the patch passed with JDK v1.7.0_95 +1 unit 7m 53s hadoop-common in the patch passed with JDK v1.8.0_91. +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.8.0_91. -1 unit 63m 36s hadoop-hdfs in the patch failed with JDK v1.8.0_91. -1 unit 7m 42s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 57m 6s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 27s Patch does not generate ASF License warnings. 208m 37s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.hdfs.TestFileAppend JDK v1.7.0_95 Failed junit tests hadoop.ipc.TestRPCWaitForProxy   hadoop.hdfs.TestHFlush Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801405/HDFS-10276.004.patch JIRA Issue HDFS-10276 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6b1b55488e29 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4a85085 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15323/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15323/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15323/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          0 mvndep 2m 38s Maven dependency ordering for branch
          +1 mvninstall 6m 35s trunk passed
          +1 compile 5m 39s trunk passed with JDK v1.8.0_91
          +1 compile 6m 38s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 6s trunk passed
          +1 mvnsite 2m 23s trunk passed
          +1 mvneclipse 0m 41s trunk passed
          +1 findbugs 5m 7s trunk passed
          +1 javadoc 2m 20s trunk passed with JDK v1.8.0_91
          +1 javadoc 3m 14s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 1m 57s the patch passed
          +1 compile 5m 36s the patch passed with JDK v1.8.0_91
          +1 javac 5m 36s the patch passed
          +1 compile 6m 39s the patch passed with JDK v1.7.0_95
          +1 javac 6m 39s the patch passed
          +1 checkstyle 1m 5s the patch passed
          +1 mvnsite 2m 21s the patch passed
          +1 mvneclipse 0m 41s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 5m 49s the patch passed
          +1 javadoc 2m 17s the patch passed with JDK v1.8.0_91
          +1 javadoc 3m 13s the patch passed with JDK v1.7.0_95
          -1 unit 6m 48s hadoop-common in the patch failed with JDK v1.8.0_91.
          +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_91.
          -1 unit 57m 11s hadoop-hdfs in the patch failed with JDK v1.8.0_91.
          +1 unit 7m 38s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 54m 0s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 26s Patch does not generate ASF License warnings.
          195m 54s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.net.TestDNS
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.TestHFlush
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801625/HDFS-10276.004.patch
          JIRA Issue HDFS-10276
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 02dece5f487c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 971af60
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15332/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15332/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. 0 mvndep 2m 38s Maven dependency ordering for branch +1 mvninstall 6m 35s trunk passed +1 compile 5m 39s trunk passed with JDK v1.8.0_91 +1 compile 6m 38s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 6s trunk passed +1 mvnsite 2m 23s trunk passed +1 mvneclipse 0m 41s trunk passed +1 findbugs 5m 7s trunk passed +1 javadoc 2m 20s trunk passed with JDK v1.8.0_91 +1 javadoc 3m 14s trunk passed with JDK v1.7.0_95 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 57s the patch passed +1 compile 5m 36s the patch passed with JDK v1.8.0_91 +1 javac 5m 36s the patch passed +1 compile 6m 39s the patch passed with JDK v1.7.0_95 +1 javac 6m 39s the patch passed +1 checkstyle 1m 5s the patch passed +1 mvnsite 2m 21s the patch passed +1 mvneclipse 0m 41s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 49s the patch passed +1 javadoc 2m 17s the patch passed with JDK v1.8.0_91 +1 javadoc 3m 13s the patch passed with JDK v1.7.0_95 -1 unit 6m 48s hadoop-common in the patch failed with JDK v1.8.0_91. +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_91. -1 unit 57m 11s hadoop-hdfs in the patch failed with JDK v1.8.0_91. +1 unit 7m 38s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 54m 0s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 195m 54s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.net.TestDNS   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.TestHFlush   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801625/HDFS-10276.004.patch JIRA Issue HDFS-10276 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 02dece5f487c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 971af60 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15332/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15332/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15332/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Colin P. McCabe Would you please take a look at the last patch? exists depends on getFileInfo, so this patch will also change the behavior of exists method. I agree with you that this patch is branch-3 only. Please let me know if you have any thought about this patch. Thanks in advance.

          Show
          yuanbo Yuanbo Liu added a comment - Colin P. McCabe Would you please take a look at the last patch? exists depends on getFileInfo , so this patch will also change the behavior of exists method. I agree with you that this patch is branch-3 only. Please let me know if you have any thought about this patch. Thanks in advance.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Arpit Agarwal Could you please help me review this issue?

          Show
          yuanbo Yuanbo Liu added a comment - Arpit Agarwal Could you please help me review this issue?
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Hi Yuanbo Liu, sorry not until later this week.

          Show
          arpitagarwal Arpit Agarwal added a comment - Hi Yuanbo Liu , sorry not until later this week.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Thanks for this, Yuanbo Liu. Sorry for the sometimes slow pace of reviews.

          If I understand correctly, the approach taken in this path is to change HDFS to throw an exception stating that the parent path is not a directory, rather than throwing an AccessControlException.

          So first of all, this sounds like an incompatible change. That's OK-- it just means this should probably go into branch-3 (trunk) rather than branch-2.

          Secondly, it seems like it would be better to make the modification inside FSPermissionChecker, rather than adding an external function. After all, this is a general problem, which affects more than just listDir. We also need to make sure that we are not giving away too much information about the filesystem. For example, if the user asks for /a/b/c, but does not have permission to list /a, we should not complain about /a/b not being a directory since that reveals privileged information.

          Show
          cmccabe Colin P. McCabe added a comment - Thanks for this, Yuanbo Liu . Sorry for the sometimes slow pace of reviews. If I understand correctly, the approach taken in this path is to change HDFS to throw an exception stating that the parent path is not a directory, rather than throwing an AccessControlException. So first of all, this sounds like an incompatible change. That's OK-- it just means this should probably go into branch-3 (trunk) rather than branch-2. Secondly, it seems like it would be better to make the modification inside FSPermissionChecker , rather than adding an external function. After all, this is a general problem, which affects more than just listDir. We also need to make sure that we are not giving away too much information about the filesystem. For example, if the user asks for /a/b/c , but does not have permission to list /a , we should not complain about /a/b not being a directory since that reveals privileged information.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Colin P. McCabe Thanks for your comment.
          When I did some research about FSPermissionChecker, I found a similar issue HDFS-5802 which would be fixed on Hadoop-2.8, and this is the key code of the patch

          if (!inodes[i].isDirectory())

          Unknown macro: { throw new AccessControlException( e.getMessage() + " (Ancestor " + inodes[i].getFullPathName() + " is not a directory)."); }

          It throws AccessControlException with "Ancestor is not a dirtectory" rather than ParentNotDirectoryException, not elegant but compatible.
          So first I'm sorry for not investigating deeply to find this issue early.
          Second, I also tested the scenario that you mentioned here

          For example, if the user asks for /a/b/c, but does not have permission to list /a, we should not complain about /a/b

          The scenario did not work as expected. So I will add a patch to fix it.
          Any comment will be appreciated, thanks in advance.

          Show
          yuanbo Yuanbo Liu added a comment - Colin P. McCabe Thanks for your comment. When I did some research about FSPermissionChecker , I found a similar issue HDFS-5802 which would be fixed on Hadoop-2.8, and this is the key code of the patch if (!inodes [i] .isDirectory()) Unknown macro: { throw new AccessControlException( e.getMessage() + " (Ancestor " + inodes[i].getFullPathName() + " is not a directory)."); } It throws AccessControlException with "Ancestor is not a dirtectory" rather than ParentNotDirectoryException , not elegant but compatible. So first I'm sorry for not investigating deeply to find this issue early. Second, I also tested the scenario that you mentioned here For example, if the user asks for /a/b/c, but does not have permission to list /a, we should not complain about /a/b The scenario did not work as expected. So I will add a patch to fix it. Any comment will be appreciated, thanks in advance.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 7s trunk passed
          +1 compile 0m 41s trunk passed with JDK v1.8.0_91
          +1 compile 0m 42s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 0m 53s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 58s trunk passed
          +1 javadoc 1m 8s trunk passed with JDK v1.8.0_91
          +1 javadoc 1m 44s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 46s the patch passed
          +1 compile 0m 36s the patch passed with JDK v1.8.0_91
          +1 javac 0m 36s the patch passed
          +1 compile 0m 40s the patch passed with JDK v1.7.0_95
          +1 javac 0m 40s the patch passed
          +1 checkstyle 0m 25s the patch passed
          +1 mvnsite 0m 48s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 6s the patch passed
          +1 javadoc 1m 2s the patch passed with JDK v1.8.0_91
          +1 javadoc 1m 43s the patch passed with JDK v1.7.0_95
          -1 unit 77m 7s hadoop-hdfs in the patch failed with JDK v1.8.0_91.
          -1 unit 84m 39s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          187m 50s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys
            hadoop.hdfs.TestAsyncDFSRename
          JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.hdfs.TestDecommissionWithStriped
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestPersistBlocks
            hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys
            hadoop.hdfs.TestAsyncDFSRename



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804150/HDFS-10276.005.patch
          JIRA Issue HDFS-10276
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c12b06d07679 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ad94411
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15442/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15442/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 7s trunk passed +1 compile 0m 41s trunk passed with JDK v1.8.0_91 +1 compile 0m 42s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 31s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 58s trunk passed +1 javadoc 1m 8s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 44s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 46s the patch passed +1 compile 0m 36s the patch passed with JDK v1.8.0_91 +1 javac 0m 36s the patch passed +1 compile 0m 40s the patch passed with JDK v1.7.0_95 +1 javac 0m 40s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 6s the patch passed +1 javadoc 1m 2s the patch passed with JDK v1.8.0_91 +1 javadoc 1m 43s the patch passed with JDK v1.7.0_95 -1 unit 77m 7s hadoop-hdfs in the patch failed with JDK v1.8.0_91. -1 unit 84m 39s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 187m 50s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys   hadoop.hdfs.TestAsyncDFSRename JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.hdfs.TestDecommissionWithStriped JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys   hadoop.hdfs.TestAsyncDFSRename Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804150/HDFS-10276.005.patch JIRA Issue HDFS-10276 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c12b06d07679 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ad94411 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15442/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15442/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Discussed with Yongjun Zhang who committed the patch of HDFS-5802, and here are the suggestions from him
          1. I prefer not to use a class member checkedAncestorIndex, instead, we can make it parameter passing.

          Change

            private void checkAncestorType(INode[] inodes, int ancestorIndex,
                AccessControlException e) throws AccessControlException {
              for (int i = 0; i <= ancestorIndex; i++) {
          

          To

           private void checkAncestorType(INode[] inodes,
                int checkedAncestorIndex, AccessControlException e)
                    throws AccessControlException {
              for (int i = 0; i <= checkedAncestorIndex; i++) {
          

          2. Change

             try {
                checkTraverse(inodeAttrs, path, ancestorIndex);
              } catch (AccessControlException e) {
                checkAncestorType(inodes, ancestorIndex, e);
              }
          

          to

             checkTraverse(inodeAttrs, inodes, path, ancestorIndex);
          

          3. Change

           private void checkTraverse(INodeAttributes[] inodes, String path, int last
                ) throws AccessControlException {
              for(int j = 0; j <= last; j++) {
                check(inodes[j], path, FsAction.EXECUTE);
              }
            }
          

          to

            private void checkTraverse(INodeAttributes[] inodeAttributes,
                INode[] inodes, String path, int last) throws AccessControlException {
              int j = 0;
              try {
                for(;j <= last; j++) {
                  check(inodeAttributes[j], path, FsAction.EXECUTE);
                }
              } catch (AccessControlException e) {
                checkAncestorType(inodes, j, e);
              }
            }
          

          4. Remove

          LOG.info("yuanbo print " + e.getMessage());
          

          Thanks a lot for Yongjun's suggestions, I uploaded a new patch for this issue.

          Show
          yuanbo Yuanbo Liu added a comment - Discussed with Yongjun Zhang who committed the patch of HDFS-5802 , and here are the suggestions from him 1. I prefer not to use a class member checkedAncestorIndex, instead, we can make it parameter passing. Change private void checkAncestorType(INode[] inodes, int ancestorIndex, AccessControlException e) throws AccessControlException { for ( int i = 0; i <= ancestorIndex; i++) { To private void checkAncestorType(INode[] inodes, int checkedAncestorIndex, AccessControlException e) throws AccessControlException { for ( int i = 0; i <= checkedAncestorIndex; i++) { 2. Change try { checkTraverse(inodeAttrs, path, ancestorIndex); } catch (AccessControlException e) { checkAncestorType(inodes, ancestorIndex, e); } to checkTraverse(inodeAttrs, inodes, path, ancestorIndex); 3. Change private void checkTraverse(INodeAttributes[] inodes, String path, int last ) throws AccessControlException { for ( int j = 0; j <= last; j++) { check(inodes[j], path, FsAction.EXECUTE); } } to private void checkTraverse(INodeAttributes[] inodeAttributes, INode[] inodes, String path, int last) throws AccessControlException { int j = 0; try { for (;j <= last; j++) { check(inodeAttributes[j], path, FsAction.EXECUTE); } } catch (AccessControlException e) { checkAncestorType(inodes, j, e); } } 4. Remove LOG.info( "yuanbo print " + e.getMessage()); Thanks a lot for Yongjun's suggestions, I uploaded a new patch for this issue.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 44s trunk passed
          +1 compile 0m 47s trunk passed
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 1m 0s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 45s trunk passed
          +1 javadoc 1m 10s trunk passed
          +1 mvninstall 0m 56s the patch passed
          +1 compile 0m 47s the patch passed
          +1 javac 0m 47s the patch passed
          +1 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 91 unchanged - 1 fixed = 91 total (was 92)
          +1 mvnsite 0m 57s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 56s the patch passed
          +1 javadoc 1m 6s the patch passed
          -1 unit 65m 18s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          86m 37s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestAsyncDFSRename



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804576/HDFS-10276.006.patch
          JIRA Issue HDFS-10276
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 14a42c31e28f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8a9ecb7
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15470/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15470/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15470/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15470/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 44s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 45s trunk passed +1 javadoc 1m 10s trunk passed +1 mvninstall 0m 56s the patch passed +1 compile 0m 47s the patch passed +1 javac 0m 47s the patch passed +1 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 91 unchanged - 1 fixed = 91 total (was 92) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 56s the patch passed +1 javadoc 1m 6s the patch passed -1 unit 65m 18s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 86m 37s Reason Tests Failed junit tests hadoop.hdfs.TestAsyncDFSRename Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804576/HDFS-10276.006.patch JIRA Issue HDFS-10276 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 14a42c31e28f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8a9ecb7 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15470/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15470/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15470/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15470/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          the failure test cases are not related to the patch

          Show
          yuanbo Yuanbo Liu added a comment - the failure test cases are not related to the patch
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks Yuanbo Liu for the new rev. I'm +1 on rev 6 and will commit by tomorrow, unless other folks have further comments.

          Show
          yzhangal Yongjun Zhang added a comment - Thanks Yuanbo Liu for the new rev. I'm +1 on rev 6 and will commit by tomorrow, unless other folks have further comments.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Committed to trunk, branch-2, branch-2.8.

          Thanks Yuanbo Liu for the contribution and other folks for the review and discussion.

          Show
          yzhangal Yongjun Zhang added a comment - Committed to trunk, branch-2, branch-2.8. Thanks Yuanbo Liu for the contribution and other folks for the review and discussion.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks Kevin Cox for reporting the issue.

          Show
          yzhangal Yongjun Zhang added a comment - Thanks Kevin Cox for reporting the issue.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9878 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9878/)
          HDFS-10276. HDFS should not expose path info that user has no permission (yzhang: rev 5ea6fd85c7aff6df28b87789f607bb57ee920639)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSPermission.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSPermissionChecker.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9878 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9878/ ) HDFS-10276 . HDFS should not expose path info that user has no permission (yzhang: rev 5ea6fd85c7aff6df28b87789f607bb57ee920639) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSPermission.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSPermissionChecker.java
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Kevin Cox Yuanbo Liu Yongjun Zhang for the work. I think it's a valid improvement for 2.7 as well; just backported it to branch-2.7.

          Show
          zhz Zhe Zhang added a comment - Thanks Kevin Cox Yuanbo Liu Yongjun Zhang for the work. I think it's a valid improvement for 2.7 as well; just backported it to branch-2.7.

            People

            • Assignee:
              yuanbo Yuanbo Liu
              Reporter:
              kevincox Kevin Cox
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development