Hadoop Common
  1. Hadoop Common
  2. HADOOP-8320

FileSystem#checkPath and AbstractFileSystem#checkPath should share code

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: fs
    • Labels:
    • Release Note:
      Hide
      Moved the checkPath logic from the AbstractFileSystem and FileSystem classes to a "checkFileSystemPath" method in the Path class. The AbstracFileSystem and FileSystem checkPath method will call the common code in checkFileSystemPath (adapter pattern). This is because the two classes aren't derived from any common ancestor.

      There are no new test cases because existing test cases test the functionality that is being refactored. Adding new test cases would be redundant.
      Show
      Moved the checkPath logic from the AbstractFileSystem and FileSystem classes to a "checkFileSystemPath" method in the Path class. The AbstracFileSystem and FileSystem checkPath method will call the common code in checkFileSystemPath (adapter pattern). This is because the two classes aren't derived from any common ancestor. There are no new test cases because existing test cases test the functionality that is being refactored. Adding new test cases would be redundant.

      Description

      Per the discussion on HADOOP-8310, these two methods can be refactored to share some code.

      1. HADOOP-8320.001.patch
        7 kB
        Vijay Bhat
      2. HADOOP-8320.002.patch
        7 kB
        Vijay Bhat
      3. HADOOP-8320.003.patch
        7 kB
        Vijay Bhat
      4. HADOOP-8320.004.patch
        7 kB
        Vijay Bhat
      5. HADOOP-8320.005.patch
        8 kB
        Vijay Bhat
      6. HADOOP-8320.007.patch
        8 kB
        Vijay Bhat

        Activity

        Hide
        Vijay Bhat added a comment -

        Aaron T. Myers, Daryn Sharp, I can take this.

        Here's how I plan to refactor the code - move the checkPath logic from the AbstractFileSystem and FileSystem classes to a "checkFileSystemPath" method in the Path class. The AbstracFileSystem and FileSystem checkPath method will call the common code in checkFileSystemPath (adapter pattern). This is because the two classes aren't derived from any common ancestor.

        However, I wanted your input of what I should do with the "canonicalizeUri" method in the FileSystem class. Since the AbstractFileSystem checkPath method should also be performing canonicalization (and it doesn't now), do you think it makes sense to move that to the Path class or is there a more logical location for it?

        Show
        Vijay Bhat added a comment - Aaron T. Myers , Daryn Sharp , I can take this. Here's how I plan to refactor the code - move the checkPath logic from the AbstractFileSystem and FileSystem classes to a "checkFileSystemPath" method in the Path class. The AbstracFileSystem and FileSystem checkPath method will call the common code in checkFileSystemPath (adapter pattern). This is because the two classes aren't derived from any common ancestor. However, I wanted your input of what I should do with the "canonicalizeUri" method in the FileSystem class. Since the AbstractFileSystem checkPath method should also be performing canonicalization (and it doesn't now), do you think it makes sense to move that to the Path class or is there a more logical location for it?
        Hide
        Vijay Bhat added a comment -

        Aaron T. Myers, Daryn Sharp I am using the getCanonicalUri from the NetUtils class to canonicalize the uri in the refactored checkPath logic.

        Show
        Vijay Bhat added a comment - Aaron T. Myers , Daryn Sharp I am using the getCanonicalUri from the NetUtils class to canonicalize the uri in the refactored checkPath logic.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12704050/HADOOP-8320.001.patch
        against trunk revision ff83ae7.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5924//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704050/HADOOP-8320.001.patch against trunk revision ff83ae7. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5924//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12704185/HADOOP-8320.001.patch
        against trunk revision ff83ae7.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5926//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704185/HADOOP-8320.001.patch against trunk revision ff83ae7. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5926//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12705231/HADOOP-8320.002.patch
        against trunk revision 968425e.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.io.file.tfile.TestTFileByteArrays
        org.apache.hadoop.fs.viewfs.TestFcPermissionsLocalFs
        org.apache.hadoop.fs.shell.TestPathData
        org.apache.hadoop.fs.TestChecksumFileSystem
        org.apache.hadoop.crypto.key.TestKeyProviderFactory
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractDelete
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractSeek
        org.apache.hadoop.fs.TestFSMainOperationsLocalFileSystem
        org.apache.hadoop.fs.TestPath
        org.apache.hadoop.util.TestGenericOptionsParser
        org.apache.hadoop.crypto.key.TestKeyShell
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractAppend
        org.apache.hadoop.fs.TestTrash
        org.apache.hadoop.fs.TestS3_LocalFileContextURI
        org.apache.hadoop.fs.TestLocalDirAllocator
        org.apache.hadoop.security.alias.TestCredShell
        org.apache.hadoop.fs.TestFileContextResolveAfs
        org.apache.hadoop.io.TestMapFile
        org.apache.hadoop.fs.shell.TestCopyPreserveFlag
        org.apache.hadoop.fs.TestFsShellCopy
        org.apache.hadoop.io.TestBloomMapFile
        org.apache.hadoop.io.TestSecureIOUtils
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractOpen
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractMkdir
        org.apache.hadoop.fs.shell.TestTextCommand
        org.apache.hadoop.fs.viewfs.TestViewFsTrash
        org.apache.hadoop.fs.viewfs.TestFcCreateMkdirLocalFs
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractMkdir
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractRename
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractDelete
        org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs
        org.apache.hadoop.fs.TestLocal_S3FileContextURI
        org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem
        org.apache.hadoop.security.ssl.TestSSLFactory
        org.apache.hadoop.io.TestSequenceFile
        org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays
        org.apache.hadoop.io.TestSequenceFileSync
        org.apache.hadoop.fs.TestLocalFSFileContextCreateMkdir
        org.apache.hadoop.fs.TestFcLocalFsUtil
        org.apache.hadoop.fs.TestStat
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractSeek
        org.apache.hadoop.fs.TestLocalFSFileContextMainOperations
        org.apache.hadoop.security.alias.TestCredentialProviderFactory
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractAppend
        org.apache.hadoop.fs.viewfs.TestChRootedFs
        org.apache.hadoop.fs.TestFileSystemCanonicalization
        org.apache.hadoop.fs.TestFsShellReturnCode
        org.apache.hadoop.fs.TestLocalFsFCStatistics
        org.apache.hadoop.fs.TestSymlinkLocalFSFileContext
        org.apache.hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractLoaded
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractOpen
        org.apache.hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs
        org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractCreate
        org.apache.hadoop.fs.TestListFiles
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractRename
        org.apache.hadoop.fs.TestFileUtil
        org.apache.hadoop.fs.TestFcLocalFsPermission
        org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays
        org.apache.hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractCreate
        org.apache.hadoop.fs.viewfs.TestChRootedFileSystem
        org.apache.hadoop.fs.viewfs.TestViewFsLocalFs
        org.apache.hadoop.fs.TestLocalFileSystem
        org.apache.hadoop.io.TestSequenceFileSerialization
        org.apache.hadoop.fs.TestFileContextDeleteOnExit
        org.apache.hadoop.io.file.tfile.TestTFileSeqFileComparison
        org.apache.hadoop.io.compress.TestCodec
        org.apache.hadoop.security.TestLdapGroupsMapping
        org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5964//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5964//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705231/HADOOP-8320.002.patch against trunk revision 968425e. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.io.file.tfile.TestTFileByteArrays org.apache.hadoop.fs.viewfs.TestFcPermissionsLocalFs org.apache.hadoop.fs.shell.TestPathData org.apache.hadoop.fs.TestChecksumFileSystem org.apache.hadoop.crypto.key.TestKeyProviderFactory org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractDelete org.apache.hadoop.fs.contract.localfs.TestLocalFSContractSeek org.apache.hadoop.fs.TestFSMainOperationsLocalFileSystem org.apache.hadoop.fs.TestPath org.apache.hadoop.util.TestGenericOptionsParser org.apache.hadoop.crypto.key.TestKeyShell org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractAppend org.apache.hadoop.fs.TestTrash org.apache.hadoop.fs.TestS3_LocalFileContextURI org.apache.hadoop.fs.TestLocalDirAllocator org.apache.hadoop.security.alias.TestCredShell org.apache.hadoop.fs.TestFileContextResolveAfs org.apache.hadoop.io.TestMapFile org.apache.hadoop.fs.shell.TestCopyPreserveFlag org.apache.hadoop.fs.TestFsShellCopy org.apache.hadoop.io.TestBloomMapFile org.apache.hadoop.io.TestSecureIOUtils org.apache.hadoop.fs.contract.localfs.TestLocalFSContractOpen org.apache.hadoop.fs.contract.localfs.TestLocalFSContractMkdir org.apache.hadoop.fs.shell.TestTextCommand org.apache.hadoop.fs.viewfs.TestViewFsTrash org.apache.hadoop.fs.viewfs.TestFcCreateMkdirLocalFs org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractMkdir org.apache.hadoop.fs.contract.localfs.TestLocalFSContractRename org.apache.hadoop.fs.contract.localfs.TestLocalFSContractDelete org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs org.apache.hadoop.fs.TestLocal_S3FileContextURI org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem org.apache.hadoop.security.ssl.TestSSLFactory org.apache.hadoop.io.TestSequenceFile org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays org.apache.hadoop.io.TestSequenceFileSync org.apache.hadoop.fs.TestLocalFSFileContextCreateMkdir org.apache.hadoop.fs.TestFcLocalFsUtil org.apache.hadoop.fs.TestStat org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractSeek org.apache.hadoop.fs.TestLocalFSFileContextMainOperations org.apache.hadoop.security.alias.TestCredentialProviderFactory org.apache.hadoop.fs.contract.localfs.TestLocalFSContractAppend org.apache.hadoop.fs.viewfs.TestChRootedFs org.apache.hadoop.fs.TestFileSystemCanonicalization org.apache.hadoop.fs.TestFsShellReturnCode org.apache.hadoop.fs.TestLocalFsFCStatistics org.apache.hadoop.fs.TestSymlinkLocalFSFileContext org.apache.hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem org.apache.hadoop.fs.contract.localfs.TestLocalFSContractLoaded org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractOpen org.apache.hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays org.apache.hadoop.fs.contract.localfs.TestLocalFSContractCreate org.apache.hadoop.fs.TestListFiles org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractRename org.apache.hadoop.fs.TestFileUtil org.apache.hadoop.fs.TestFcLocalFsPermission org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays org.apache.hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractCreate org.apache.hadoop.fs.viewfs.TestChRootedFileSystem org.apache.hadoop.fs.viewfs.TestViewFsLocalFs org.apache.hadoop.fs.TestLocalFileSystem org.apache.hadoop.io.TestSequenceFileSerialization org.apache.hadoop.fs.TestFileContextDeleteOnExit org.apache.hadoop.io.file.tfile.TestTFileSeqFileComparison org.apache.hadoop.io.compress.TestCodec org.apache.hadoop.security.TestLdapGroupsMapping org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5964//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5964//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12705432/HADOOP-8320.003.patch
        against trunk revision f0dea03.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.fs.shell.TestPathData
        org.apache.hadoop.fs.TestFSMainOperationsLocalFileSystem
        org.apache.hadoop.fs.TestLocalDirAllocator
        org.apache.hadoop.fs.shell.TestCopyPreserveFlag
        org.apache.hadoop.fs.TestFsShellCopy
        org.apache.hadoop.io.TestSecureIOUtils
        org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem
        org.apache.hadoop.fs.TestFileSystemCanonicalization
        org.apache.hadoop.fs.viewfs.TestChRootedFileSystem
        org.apache.hadoop.fs.TestLocalFileSystem
        org.apache.hadoop.io.compress.TestCodec
        org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5966//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5966//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705432/HADOOP-8320.003.patch against trunk revision f0dea03. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.shell.TestPathData org.apache.hadoop.fs.TestFSMainOperationsLocalFileSystem org.apache.hadoop.fs.TestLocalDirAllocator org.apache.hadoop.fs.shell.TestCopyPreserveFlag org.apache.hadoop.fs.TestFsShellCopy org.apache.hadoop.io.TestSecureIOUtils org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem org.apache.hadoop.fs.TestFileSystemCanonicalization org.apache.hadoop.fs.viewfs.TestChRootedFileSystem org.apache.hadoop.fs.TestLocalFileSystem org.apache.hadoop.io.compress.TestCodec org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5966//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5966//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12705776/HADOOP-8320.004.patch
        against trunk revision e37ca22.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.fs.TestAfsCheckPath
        org.apache.hadoop.fs.TestFileSystemCanonicalization

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5974//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5974//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5974//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705776/HADOOP-8320.004.patch against trunk revision e37ca22. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.TestAfsCheckPath org.apache.hadoop.fs.TestFileSystemCanonicalization Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5974//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5974//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5974//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12708691/HADOOP-8320.005.patch
        against trunk revision 2e79f1c.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.fs.TestFcLocalFsUtil
        org.apache.hadoop.fs.TestLocalDirAllocator
        org.apache.hadoop.io.file.tfile.TestTFileSeqFileComparison
        org.apache.hadoop.fs.shell.TestCopy
        org.apache.hadoop.util.TestDiskChecker
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractAppend
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractRename
        org.apache.hadoop.fs.TestFileUtil
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractOpen
        org.apache.hadoop.security.ssl.TestSSLFactory
        org.apache.hadoop.fs.TestFsShellCopy
        org.apache.hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem
        org.apache.hadoop.fs.TestLocalFileSystem
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractDelete
        org.apache.hadoop.fs.shell.find.TestIname
        org.apache.hadoop.fs.TestFSMainOperationsLocalFileSystem
        org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem
        org.apache.hadoop.io.file.tfile.TestTFileStreams
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractOpen
        org.apache.hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs
        org.apache.hadoop.util.TestGenericOptionsParser
        org.apache.hadoop.io.TestSetFile
        org.apache.hadoop.fs.TestS3_LocalFileContextURI
        org.apache.hadoop.crypto.key.TestKeyShell
        org.apache.hadoop.fs.viewfs.TestViewFileSystemDelegationTokenSupport
        org.apache.hadoop.fs.viewfs.TestViewFileSystemDelegation
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractMkdir
        org.apache.hadoop.io.compress.TestCodec
        org.apache.hadoop.fs.TestGetFileBlockLocations
        org.apache.hadoop.fs.viewfs.TestViewFsLocalFs
        org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsStreams
        org.apache.hadoop.fs.TestChecksumFileSystem
        org.apache.hadoop.fs.TestTruncatedInputBug
        org.apache.hadoop.security.alias.TestCredShell
        org.apache.hadoop.security.alias.TestCredentialProviderFactory
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractCreate
        org.apache.hadoop.io.TestMapFile
        org.apache.hadoop.fs.TestPath
        org.apache.hadoop.fs.viewfs.TestChRootedFs
        org.apache.hadoop.security.TestLdapGroupsMapping
        org.apache.hadoop.fs.TestFilterFileSystem
        org.apache.hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem
        org.apache.hadoop.fs.shell.TestPathData
        org.apache.hadoop.fs.TestFileSystemInitialization
        org.apache.hadoop.fs.shell.find.TestPrint0
        org.apache.hadoop.io.TestArrayFile
        org.apache.hadoop.fs.TestLocalFsFCStatistics
        org.apache.hadoop.fs.TestLocalFileSystemPermission
        org.apache.hadoop.fs.TestListFiles
        org.apache.hadoop.fs.shell.find.TestFind
        org.apache.hadoop.io.file.tfile.TestTFileByteArrays
        org.apache.hadoop.io.file.tfile.TestTFileSplit
        org.apache.hadoop.io.file.tfile.TestTFileSeek
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractMkdir
        org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays
        org.apache.hadoop.fs.TestFsShellReturnCode
        org.apache.hadoop.io.file.tfile.TestVLong
        org.apache.hadoop.fs.TestHarFileSystemBasics
        org.apache.hadoop.fs.TestTrash
        org.apache.hadoop.crypto.key.TestKeyProviderFactory
        org.apache.hadoop.fs.shell.TestLs
        org.apache.hadoop.fs.shell.TestMove
        org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays
        org.apache.hadoop.io.file.tfile.TestTFileComparators
        org.apache.hadoop.fs.shell.find.TestName
        org.apache.hadoop.util.TestNodeHealthScriptRunner
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractAppend
        org.apache.hadoop.io.TestSequenceFileSync
        org.apache.hadoop.fs.viewfs.TestChRootedFileSystem
        org.apache.hadoop.io.file.tfile.TestTFileUnsortedByteArrays
        org.apache.hadoop.fs.TestLocalFSFileContextMainOperations
        org.apache.hadoop.fs.TestAvroFSInput
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractRename
        org.apache.hadoop.fs.viewfs.TestFcPermissionsLocalFs
        org.apache.hadoop.fs.shell.TestCount
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractSeek
        org.apache.hadoop.fs.TestSymlinkLocalFSFileContext
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractLoaded
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractCreate
        org.apache.hadoop.fs.viewfs.TestViewFsTrash
        org.apache.hadoop.fs.viewfs.TestViewfsFileStatus
        org.apache.hadoop.io.file.tfile.TestTFileComparator2
        org.apache.hadoop.fs.TestFcLocalFsPermission
        org.apache.hadoop.fs.shell.TestCopyPreserveFlag
        org.apache.hadoop.fs.TestLocalFSFileContextCreateMkdir
        org.apache.hadoop.fs.shell.find.TestPrint
        org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem
        org.apache.hadoop.fs.TestStat
        org.apache.hadoop.fs.TestHarFileSystem
        org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs
        org.apache.hadoop.fs.contract.localfs.TestLocalFSContractDelete
        org.apache.hadoop.io.TestBloomMapFile
        org.apache.hadoop.fs.TestLocal_S3FileContextURI
        org.apache.hadoop.io.TestSequenceFileSerialization
        org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays
        org.apache.hadoop.io.nativeio.TestNativeIO
        org.apache.hadoop.fs.shell.TestAclCommands
        org.apache.hadoop.io.TestSecureIOUtils
        org.apache.hadoop.io.TestSequenceFile
        org.apache.hadoop.io.file.tfile.TestTFile
        org.apache.hadoop.fs.TestFileContextDeleteOnExit
        org.apache.hadoop.fs.viewfs.TestViewFsURIs
        org.apache.hadoop.fs.viewfs.TestFcCreateMkdirLocalFs
        org.apache.hadoop.fs.TestFileContextResolveAfs
        org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractSeek
        org.apache.hadoop.crypto.TestCryptoStreamsForLocalFS
        org.apache.hadoop.fs.shell.TestTextCommand
        org.apache.hadoop.conf.TestConfiguration

        The following test timeouts occurred in hadoop-common-project/hadoop-common:

        org.apache.hadoop.fs.TestFileSystemCaching

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/6038//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/6038//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12708691/HADOOP-8320.005.patch against trunk revision 2e79f1c. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.TestFcLocalFsUtil org.apache.hadoop.fs.TestLocalDirAllocator org.apache.hadoop.io.file.tfile.TestTFileSeqFileComparison org.apache.hadoop.fs.shell.TestCopy org.apache.hadoop.util.TestDiskChecker org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractAppend org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractRename org.apache.hadoop.fs.TestFileUtil org.apache.hadoop.fs.contract.localfs.TestLocalFSContractOpen org.apache.hadoop.security.ssl.TestSSLFactory org.apache.hadoop.fs.TestFsShellCopy org.apache.hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem org.apache.hadoop.fs.TestLocalFileSystem org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractDelete org.apache.hadoop.fs.shell.find.TestIname org.apache.hadoop.fs.TestFSMainOperationsLocalFileSystem org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem org.apache.hadoop.io.file.tfile.TestTFileStreams org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractOpen org.apache.hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs org.apache.hadoop.util.TestGenericOptionsParser org.apache.hadoop.io.TestSetFile org.apache.hadoop.fs.TestS3_LocalFileContextURI org.apache.hadoop.crypto.key.TestKeyShell org.apache.hadoop.fs.viewfs.TestViewFileSystemDelegationTokenSupport org.apache.hadoop.fs.viewfs.TestViewFileSystemDelegation org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractMkdir org.apache.hadoop.io.compress.TestCodec org.apache.hadoop.fs.TestGetFileBlockLocations org.apache.hadoop.fs.viewfs.TestViewFsLocalFs org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsStreams org.apache.hadoop.fs.TestChecksumFileSystem org.apache.hadoop.fs.TestTruncatedInputBug org.apache.hadoop.security.alias.TestCredShell org.apache.hadoop.security.alias.TestCredentialProviderFactory org.apache.hadoop.fs.contract.localfs.TestLocalFSContractCreate org.apache.hadoop.io.TestMapFile org.apache.hadoop.fs.TestPath org.apache.hadoop.fs.viewfs.TestChRootedFs org.apache.hadoop.security.TestLdapGroupsMapping org.apache.hadoop.fs.TestFilterFileSystem org.apache.hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem org.apache.hadoop.fs.shell.TestPathData org.apache.hadoop.fs.TestFileSystemInitialization org.apache.hadoop.fs.shell.find.TestPrint0 org.apache.hadoop.io.TestArrayFile org.apache.hadoop.fs.TestLocalFsFCStatistics org.apache.hadoop.fs.TestLocalFileSystemPermission org.apache.hadoop.fs.TestListFiles org.apache.hadoop.fs.shell.find.TestFind org.apache.hadoop.io.file.tfile.TestTFileByteArrays org.apache.hadoop.io.file.tfile.TestTFileSplit org.apache.hadoop.io.file.tfile.TestTFileSeek org.apache.hadoop.fs.contract.localfs.TestLocalFSContractMkdir org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays org.apache.hadoop.fs.TestFsShellReturnCode org.apache.hadoop.io.file.tfile.TestVLong org.apache.hadoop.fs.TestHarFileSystemBasics org.apache.hadoop.fs.TestTrash org.apache.hadoop.crypto.key.TestKeyProviderFactory org.apache.hadoop.fs.shell.TestLs org.apache.hadoop.fs.shell.TestMove org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsJClassComparatorByteArrays org.apache.hadoop.io.file.tfile.TestTFileComparators org.apache.hadoop.fs.shell.find.TestName org.apache.hadoop.util.TestNodeHealthScriptRunner org.apache.hadoop.fs.contract.localfs.TestLocalFSContractAppend org.apache.hadoop.io.TestSequenceFileSync org.apache.hadoop.fs.viewfs.TestChRootedFileSystem org.apache.hadoop.io.file.tfile.TestTFileUnsortedByteArrays org.apache.hadoop.fs.TestLocalFSFileContextMainOperations org.apache.hadoop.fs.TestAvroFSInput org.apache.hadoop.fs.contract.localfs.TestLocalFSContractRename org.apache.hadoop.fs.viewfs.TestFcPermissionsLocalFs org.apache.hadoop.fs.shell.TestCount org.apache.hadoop.fs.contract.localfs.TestLocalFSContractSeek org.apache.hadoop.fs.TestSymlinkLocalFSFileContext org.apache.hadoop.fs.contract.localfs.TestLocalFSContractLoaded org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractCreate org.apache.hadoop.fs.viewfs.TestViewFsTrash org.apache.hadoop.fs.viewfs.TestViewfsFileStatus org.apache.hadoop.io.file.tfile.TestTFileComparator2 org.apache.hadoop.fs.TestFcLocalFsPermission org.apache.hadoop.fs.shell.TestCopyPreserveFlag org.apache.hadoop.fs.TestLocalFSFileContextCreateMkdir org.apache.hadoop.fs.shell.find.TestPrint org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem org.apache.hadoop.fs.TestStat org.apache.hadoop.fs.TestHarFileSystem org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs org.apache.hadoop.fs.contract.localfs.TestLocalFSContractDelete org.apache.hadoop.io.TestBloomMapFile org.apache.hadoop.fs.TestLocal_S3FileContextURI org.apache.hadoop.io.TestSequenceFileSerialization org.apache.hadoop.io.file.tfile.TestTFileNoneCodecsByteArrays org.apache.hadoop.io.nativeio.TestNativeIO org.apache.hadoop.fs.shell.TestAclCommands org.apache.hadoop.io.TestSecureIOUtils org.apache.hadoop.io.TestSequenceFile org.apache.hadoop.io.file.tfile.TestTFile org.apache.hadoop.fs.TestFileContextDeleteOnExit org.apache.hadoop.fs.viewfs.TestViewFsURIs org.apache.hadoop.fs.viewfs.TestFcCreateMkdirLocalFs org.apache.hadoop.fs.TestFileContextResolveAfs org.apache.hadoop.fs.contract.rawlocal.TestRawlocalContractSeek org.apache.hadoop.crypto.TestCryptoStreamsForLocalFS org.apache.hadoop.fs.shell.TestTextCommand org.apache.hadoop.conf.TestConfiguration The following test timeouts occurred in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.TestFileSystemCaching Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/6038//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/6038//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12708789/HADOOP-8320.007.patch
        against trunk revision c94d594.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/6044//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/6044//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12708789/HADOOP-8320.007.patch against trunk revision c94d594. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/6044//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/6044//console This message is automatically generated.
        Hide
        Vijay Bhat added a comment -

        Aaron T. Myers all test cases now finally pass for the refactor. There are no new additional test cases since existing ones capture the functionality of the code being refactored. Would appreciate a review when you get a chance.

        Show
        Vijay Bhat added a comment - Aaron T. Myers all test cases now finally pass for the refactor. There are no new additional test cases since existing ones capture the functionality of the code being refactored. Would appreciate a review when you get a chance.
        Hide
        Vinayakumar B added a comment -

        Vijay Bhat, thanks for the update on patch.

        Two things need to be corrected.

        1. AbstractFileSystem doesn't support relative paths, Instead it supports only fully qualified or paths starting with root '/'. Its okay untill relative path is corrected already in FileContext before passing to AFS.
        1. Usage of NetUtils.getCanonicalUri(uri, getDefaultPort()); was intentionally not done, because there was no need of DNS replacement for the host, which might be time consuming, instead just needed to replace default port. Check javadoc in FileSystem#canonicalizeUri()
        Show
        Vinayakumar B added a comment - Vijay Bhat , thanks for the update on patch. Two things need to be corrected. AbstractFileSystem doesn't support relative paths, Instead it supports only fully qualified or paths starting with root '/'. Its okay untill relative path is corrected already in FileContext before passing to AFS. Usage of NetUtils.getCanonicalUri(uri, getDefaultPort()); was intentionally not done, because there was no need of DNS replacement for the host, which might be time consuming, instead just needed to replace default port. Check javadoc in FileSystem#canonicalizeUri()
        Hide
        Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12708789/HADOOP-8320.007.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / e830207
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6916/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12708789/HADOOP-8320.007.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e830207 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6916/console This message was automatically generated.

          People

          • Assignee:
            Vijay Bhat
            Reporter:
            Aaron T. Myers
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development