Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9110

Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for better efficiency

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      This is a request to do some cosmetic improvements on top of HDFS-8480. There a couple of File -> java.nio.file.Path conversions which is a little bit distracting.

      The second aspect is more around efficiency, to be perfectly honest I'm not sure what the number of files that may be processed. However as HDFS-8480 eludes to it appears that this number could be significantly large.

      The current implementation is basically a collect and process where all files first is being examined; put into a collection and after that processed. HDFS-8480 could simply be further enhanced by employing a single iteration without creating an intermediary collection of filenames by using a FileWalker

      1. HDFS-9110.00.patch
        2 kB
        Charlie Helin
      2. HDFS-9110.01.patch
        2 kB
        Charlie Helin
      3. HDFS-9110.02.patch
        2 kB
        Charlie Helin
      4. HDFS-9110.03.patch
        2 kB
        Charlie Helin
      5. HDFS-9110.04.patch
        3 kB
        Charlie Helin
      6. HDFS-9110.05.patch
        3 kB
        Charlie Helin
      7. HDFS-9110.06.patch
        3 kB
        Charlie Helin
      8. HDFS-9110.07.patch
        3 kB
        Charlie Helin
      9. HDFS-9110.08.patch
        3 kB
        Charlie Helin

        Activity

        Hide
        chelin Charlie Helin added a comment -

        Suggested solution

        Show
        chelin Charlie Helin added a comment - Suggested solution
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 34s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 49s There were no new javac warning messages.
        +1 javadoc 10m 4s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 19s The applied patch generated 5 new checkstyle issues (total was 2, now 6).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 11s Pre-build of native portion
        -1 hdfs tests 189m 7s Tests failed in hadoop-hdfs.
            234m 4s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestDFSUpgradeFromImage



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761192/HDFS-9110.00.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / e3ace31
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12557/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12557/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12557/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12557/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 34s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 49s There were no new javac warning messages. +1 javadoc 10m 4s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 19s The applied patch generated 5 new checkstyle issues (total was 2, now 6). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 11s Pre-build of native portion -1 hdfs tests 189m 7s Tests failed in hadoop-hdfs.     234m 4s   Reason Tests Failed unit tests hadoop.hdfs.TestDFSUpgradeFromImage Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761192/HDFS-9110.00.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e3ace31 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12557/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12557/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12557/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12557/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 33s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 45s There were no new javac warning messages.
        +1 javadoc 10m 8s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 19s The applied patch generated 5 new checkstyle issues (total was 2, now 6).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 28s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 17s Pre-build of native portion
        -1 hdfs tests 147m 20s Tests failed in hadoop-hdfs.
            192m 17s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestParallelShortCircuitRead
          hadoop.fs.contract.hdfs.TestHDFSContractMkdir
          hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
          hadoop.hdfs.TestBlockStoragePolicy
          hadoop.cli.TestDeleteCLI
          hadoop.hdfs.TestFileLengthOnClusterRestart
          hadoop.hdfs.TestAppendSnapshotTruncate
          hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory
          hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
          hadoop.cli.TestHDFSCLI
          hadoop.hdfs.TestDFSUpgradeFromImage
          hadoop.hdfs.TestReplaceDatanodeOnFailure
          hadoop.hdfs.TestRemoteBlockReader2
          hadoop.hdfs.server.namenode.TestStartup
          hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
          hadoop.hdfs.TestDFSStorageStateRecovery
          hadoop.hdfs.server.namenode.TestFSImageWithXAttr
          hadoop.hdfs.TestRemoteBlockReader
          hadoop.hdfs.TestMultiThreadedHflush
          hadoop.fs.contract.hdfs.TestHDFSContractRename
          hadoop.hdfs.TestBlockReaderLocal
          hadoop.cli.TestCacheAdminCLI
          hadoop.hdfs.server.mover.TestMover
          hadoop.fs.loadGenerator.TestLoadGenerator
          hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
          hadoop.fs.TestFcHdfsSetUMask
          hadoop.hdfs.TestPread
          hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
          hadoop.hdfs.crypto.TestHdfsCryptoStreams
          hadoop.fs.viewfs.TestViewFsFileStatusHdfs
          hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl
          hadoop.hdfs.TestDFSAddressConfig
          hadoop.tracing.TestTracingShortCircuitLocalRead
          hadoop.tracing.TestTracing
          hadoop.hdfs.TestParallelRead
          hadoop.hdfs.TestRestartDFS
          hadoop.fs.TestWebHdfsFileContextMainOperations
          hadoop.hdfs.TestParallelShortCircuitReadNoChecksum
          hadoop.hdfs.TestParallelShortCircuitLegacyRead
          hadoop.net.TestNetworkTopology
          hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
          hadoop.fs.TestFcHdfsCreateMkdir
          hadoop.hdfs.TestHFlush
          hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
          hadoop.hdfs.TestFetchImage
          hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
          hadoop.hdfs.server.namenode.ha.TestDNFencing
          hadoop.hdfs.TestDFSUpgrade
          hadoop.fs.TestFcHdfsPermission
          hadoop.hdfs.TestMissingBlocksAlert
          hadoop.hdfs.server.namenode.ha.TestHAMetrics
          hadoop.hdfs.TestQuota
          hadoop.hdfs.server.namenode.TestSaveNamespace
          hadoop.hdfs.tools.TestDebugAdmin
          hadoop.hdfs.TestPersistBlocks
          hadoop.hdfs.server.namenode.TestSnapshotPathINodes
          hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
          hadoop.hdfs.TestModTime
          hadoop.hdfs.server.blockmanagement.TestBlockManager
          hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA
          hadoop.hdfs.TestGetBlocks
          hadoop.hdfs.TestDFSStartupVersions
          hadoop.hdfs.TestRenameWhileOpen
          hadoop.fs.viewfs.TestViewFsWithXAttrs
          hadoop.tools.TestJMXGet
          hadoop.hdfs.TestFileAppend4
          hadoop.TestGenericRefresh
          hadoop.hdfs.server.namenode.TestAddBlock
          hadoop.hdfs.TestDataTransferKeepalive
          hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
          hadoop.hdfs.TestFileAppend2
          hadoop.hdfs.server.namenode.TestAuditLogs
          hadoop.fs.TestSWebHdfsFileContextMainOperations
          hadoop.hdfs.TestInjectionForSimulatedStorage
          hadoop.fs.contract.hdfs.TestHDFSContractDelete
          hadoop.fs.contract.hdfs.TestHDFSContractSetTimes
          hadoop.hdfs.TestLocalDFS
          hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
          hadoop.hdfs.TestCrcCorruption
          hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForXAttr
          hadoop.hdfs.TestConnCache
          hadoop.hdfs.TestGetFileChecksum
          hadoop.hdfs.TestFileAppend
          hadoop.hdfs.TestReservedRawPaths
          hadoop.TestRefreshCallQueue
          hadoop.hdfs.TestDisableConnCache
          hadoop.cli.TestXAttrCLI
          hadoop.hdfs.TestParallelShortCircuitReadUnCached
          hadoop.fs.TestUrlStreamHandler
          hadoop.fs.TestEnhancedByteBufferAccess
          hadoop.hdfs.server.namenode.TestAuditLogger
          hadoop.hdfs.TestRollingUpgrade
          hadoop.fs.viewfs.TestViewFileSystemWithXAttrs
          hadoop.fs.TestHDFSFileContextMainOperations
          hadoop.hdfs.TestSafeMode
          hadoop.hdfs.protocolPB.TestPBHelper
          hadoop.hdfs.server.namenode.TestBackupNode
          hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
          hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
          hadoop.hdfs.server.namenode.TestAclConfigFlag
          hadoop.hdfs.TestWriteRead
          hadoop.hdfs.TestExternalBlockReader
          hadoop.fs.contract.hdfs.TestHDFSContractAppend
          hadoop.fs.contract.hdfs.TestHDFSContractOpen
          hadoop.hdfs.TestFileCreation
          hadoop.hdfs.server.mover.TestStorageMover
          hadoop.hdfs.TestClientReportBadBlock
          hadoop.tracing.TestTraceAdmin
          hadoop.hdfs.server.namenode.TestBlockUnderConstruction
          hadoop.hdfs.TestFileCorruption
          hadoop.hdfs.server.namenode.TestHDFSConcat
          hadoop.hdfs.TestClose
          hadoop.hdfs.TestAbandonBlock
          hadoop.hdfs.TestDFSInputStream
          hadoop.hdfs.TestLeaseRecovery
          hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
          hadoop.hdfs.tools.TestDFSAdmin
          hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
          hadoop.hdfs.TestWriteBlockGetsBlockLengthHint
          hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
          hadoop.fs.viewfs.TestViewFsHdfs
          hadoop.fs.contract.hdfs.TestHDFSContractConcat
          hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
          hadoop.hdfs.TestHDFSTrash
          hadoop.hdfs.TestSmallBlock
          hadoop.cli.TestCryptoAdminCLI
          hadoop.hdfs.TestFileCreationDelete
          hadoop.hdfs.TestReplication
          hadoop.hdfs.server.namenode.ha.TestHAAppend
          hadoop.cli.TestAclCLI
          hadoop.fs.viewfs.TestViewFileSystemWithAcls
          hadoop.fs.TestSymlinkHdfsFileContext
          hadoop.fs.TestSymlinkHdfsFileSystem
          hadoop.hdfs.TestFileAppend3
          hadoop.fs.TestUnbuffer
          hadoop.hdfs.TestPipelines
          hadoop.hdfs.server.namenode.ha.TestHASafeMode
          hadoop.hdfs.qjournal.TestSecureNNWithQJM
          hadoop.hdfs.TestBlockReaderFactory
          hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.TestFSOutputSummer
          hadoop.hdfs.TestBlockReaderLocalLegacy
          hadoop.fs.viewfs.TestViewFsWithAcls
          hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
          hadoop.hdfs.TestEncryptionZones
          hadoop.fs.TestResolveHdfsSymlink
          hadoop.hdfs.TestFileConcurrentReader
          hadoop.fs.viewfs.TestViewFsDefaultValue
          hadoop.hdfs.TestAppendDifferentChecksum
          hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus
          hadoop.hdfs.server.namenode.TestCreateEditsLog
          hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA
          hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary
          hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
          hadoop.fs.viewfs.TestViewFileSystemHdfs
          hadoop.hdfs.TestParallelUnixDomainRead
          hadoop.hdfs.tools.TestStoragePolicyCommands
          hadoop.hdfs.TestDFSRemove
          hadoop.fs.viewfs.TestViewFsAtHdfsRoot
          hadoop.hdfs.TestFileCreationClient
          hadoop.fs.contract.hdfs.TestHDFSContractCreate
          hadoop.fs.permission.TestStickyBit
          hadoop.hdfs.TestEncryptionZonesWithKMS
          hadoop.fs.TestSymlinkHdfsDisable
          hadoop.hdfs.server.namenode.TestDeleteRace
          hadoop.hdfs.TestLease
          hadoop.hdfs.TestLargeBlock
          hadoop.fs.TestGlobPaths
          hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes
          hadoop.hdfs.TestClientProtocolForPipelineRecovery
          hadoop.fs.contract.hdfs.TestHDFSContractSeek
          hadoop.fs.shell.TestHdfsTextCommand
          hadoop.hdfs.TestBlockMissingException



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761309/HDFS-9110.01.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / c39ddc3
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12559/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12559/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12559/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12559/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 33s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 45s There were no new javac warning messages. +1 javadoc 10m 8s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 19s The applied patch generated 5 new checkstyle issues (total was 2, now 6). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 17s Pre-build of native portion -1 hdfs tests 147m 20s Tests failed in hadoop-hdfs.     192m 17s   Reason Tests Failed unit tests hadoop.hdfs.TestParallelShortCircuitRead   hadoop.fs.contract.hdfs.TestHDFSContractMkdir   hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens   hadoop.hdfs.TestBlockStoragePolicy   hadoop.cli.TestDeleteCLI   hadoop.hdfs.TestFileLengthOnClusterRestart   hadoop.hdfs.TestAppendSnapshotTruncate   hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate   hadoop.cli.TestHDFSCLI   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.TestRemoteBlockReader2   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics   hadoop.hdfs.TestDFSStorageStateRecovery   hadoop.hdfs.server.namenode.TestFSImageWithXAttr   hadoop.hdfs.TestRemoteBlockReader   hadoop.hdfs.TestMultiThreadedHflush   hadoop.fs.contract.hdfs.TestHDFSContractRename   hadoop.hdfs.TestBlockReaderLocal   hadoop.cli.TestCacheAdminCLI   hadoop.hdfs.server.mover.TestMover   hadoop.fs.loadGenerator.TestLoadGenerator   hadoop.hdfs.server.namenode.TestLargeDirectoryDelete   hadoop.fs.TestFcHdfsSetUMask   hadoop.hdfs.TestPread   hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   hadoop.hdfs.crypto.TestHdfsCryptoStreams   hadoop.fs.viewfs.TestViewFsFileStatusHdfs   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl   hadoop.hdfs.TestDFSAddressConfig   hadoop.tracing.TestTracingShortCircuitLocalRead   hadoop.tracing.TestTracing   hadoop.hdfs.TestParallelRead   hadoop.hdfs.TestRestartDFS   hadoop.fs.TestWebHdfsFileContextMainOperations   hadoop.hdfs.TestParallelShortCircuitReadNoChecksum   hadoop.hdfs.TestParallelShortCircuitLegacyRead   hadoop.net.TestNetworkTopology   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   hadoop.fs.TestFcHdfsCreateMkdir   hadoop.hdfs.TestHFlush   hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled   hadoop.hdfs.TestFetchImage   hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication   hadoop.hdfs.server.namenode.ha.TestDNFencing   hadoop.hdfs.TestDFSUpgrade   hadoop.fs.TestFcHdfsPermission   hadoop.hdfs.TestMissingBlocksAlert   hadoop.hdfs.server.namenode.ha.TestHAMetrics   hadoop.hdfs.TestQuota   hadoop.hdfs.server.namenode.TestSaveNamespace   hadoop.hdfs.tools.TestDebugAdmin   hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.server.namenode.TestSnapshotPathINodes   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer   hadoop.hdfs.TestModTime   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA   hadoop.hdfs.TestGetBlocks   hadoop.hdfs.TestDFSStartupVersions   hadoop.hdfs.TestRenameWhileOpen   hadoop.fs.viewfs.TestViewFsWithXAttrs   hadoop.tools.TestJMXGet   hadoop.hdfs.TestFileAppend4   hadoop.TestGenericRefresh   hadoop.hdfs.server.namenode.TestAddBlock   hadoop.hdfs.TestDataTransferKeepalive   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.TestFileAppend2   hadoop.hdfs.server.namenode.TestAuditLogs   hadoop.fs.TestSWebHdfsFileContextMainOperations   hadoop.hdfs.TestInjectionForSimulatedStorage   hadoop.fs.contract.hdfs.TestHDFSContractDelete   hadoop.fs.contract.hdfs.TestHDFSContractSetTimes   hadoop.hdfs.TestLocalDFS   hadoop.hdfs.server.namenode.TestListCorruptFileBlocks   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForXAttr   hadoop.hdfs.TestConnCache   hadoop.hdfs.TestGetFileChecksum   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestReservedRawPaths   hadoop.TestRefreshCallQueue   hadoop.hdfs.TestDisableConnCache   hadoop.cli.TestXAttrCLI   hadoop.hdfs.TestParallelShortCircuitReadUnCached   hadoop.fs.TestUrlStreamHandler   hadoop.fs.TestEnhancedByteBufferAccess   hadoop.hdfs.server.namenode.TestAuditLogger   hadoop.hdfs.TestRollingUpgrade   hadoop.fs.viewfs.TestViewFileSystemWithXAttrs   hadoop.fs.TestHDFSFileContextMainOperations   hadoop.hdfs.TestSafeMode   hadoop.hdfs.protocolPB.TestPBHelper   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement   hadoop.hdfs.server.namenode.TestAclConfigFlag   hadoop.hdfs.TestWriteRead   hadoop.hdfs.TestExternalBlockReader   hadoop.fs.contract.hdfs.TestHDFSContractAppend   hadoop.fs.contract.hdfs.TestHDFSContractOpen   hadoop.hdfs.TestFileCreation   hadoop.hdfs.server.mover.TestStorageMover   hadoop.hdfs.TestClientReportBadBlock   hadoop.tracing.TestTraceAdmin   hadoop.hdfs.server.namenode.TestBlockUnderConstruction   hadoop.hdfs.TestFileCorruption   hadoop.hdfs.server.namenode.TestHDFSConcat   hadoop.hdfs.TestClose   hadoop.hdfs.TestAbandonBlock   hadoop.hdfs.TestDFSInputStream   hadoop.hdfs.TestLeaseRecovery   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.tools.TestDFSAdmin   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.TestWriteBlockGetsBlockLengthHint   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   hadoop.fs.viewfs.TestViewFsHdfs   hadoop.fs.contract.hdfs.TestHDFSContractConcat   hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot   hadoop.hdfs.TestHDFSTrash   hadoop.hdfs.TestSmallBlock   hadoop.cli.TestCryptoAdminCLI   hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.TestReplication   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.cli.TestAclCLI   hadoop.fs.viewfs.TestViewFileSystemWithAcls   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.fs.TestSymlinkHdfsFileSystem   hadoop.hdfs.TestFileAppend3   hadoop.fs.TestUnbuffer   hadoop.hdfs.TestPipelines   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.TestBlockReaderFactory   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestFSOutputSummer   hadoop.hdfs.TestBlockReaderLocalLegacy   hadoop.fs.viewfs.TestViewFsWithAcls   hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   hadoop.hdfs.TestEncryptionZones   hadoop.fs.TestResolveHdfsSymlink   hadoop.hdfs.TestFileConcurrentReader   hadoop.fs.viewfs.TestViewFsDefaultValue   hadoop.hdfs.TestAppendDifferentChecksum   hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus   hadoop.hdfs.server.namenode.TestCreateEditsLog   hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary   hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer   hadoop.fs.viewfs.TestViewFileSystemHdfs   hadoop.hdfs.TestParallelUnixDomainRead   hadoop.hdfs.tools.TestStoragePolicyCommands   hadoop.hdfs.TestDFSRemove   hadoop.fs.viewfs.TestViewFsAtHdfsRoot   hadoop.hdfs.TestFileCreationClient   hadoop.fs.contract.hdfs.TestHDFSContractCreate   hadoop.fs.permission.TestStickyBit   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.fs.TestSymlinkHdfsDisable   hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.TestLease   hadoop.hdfs.TestLargeBlock   hadoop.fs.TestGlobPaths   hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.fs.contract.hdfs.TestHDFSContractSeek   hadoop.fs.shell.TestHdfsTextCommand   hadoop.hdfs.TestBlockMissingException Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761309/HDFS-9110.01.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c39ddc3 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12559/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12559/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12559/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12559/console This message was automatically generated.
        Hide
        chelin Charlie Helin added a comment -

        No changes, but prior build seems off.

        Show
        chelin Charlie Helin added a comment - No changes, but prior build seems off.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 48s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 51s There were no new javac warning messages.
        +1 javadoc 9m 58s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 19s The applied patch generated 5 new checkstyle issues (total was 2, now 6).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 31s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 10s Pre-build of native portion
        -1 hdfs tests 194m 17s Tests failed in hadoop-hdfs.
            239m 19s  



        Reason Tests
        Failed unit tests hadoop.cli.TestHDFSCLI
          hadoop.hdfs.TestReplaceDatanodeOnFailure
          hadoop.hdfs.server.blockmanagement.TestBlockManager
          hadoop.TestGenericRefresh
          hadoop.cli.TestAclCLI
          hadoop.hdfs.server.namenode.TestFileTruncate



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761437/HDFS-9110.02.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / c9cb6a5
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12567/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12567/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12567/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12567/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 48s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 51s There were no new javac warning messages. +1 javadoc 9m 58s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 19s The applied patch generated 5 new checkstyle issues (total was 2, now 6). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 26s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 10s Pre-build of native portion -1 hdfs tests 194m 17s Tests failed in hadoop-hdfs.     239m 19s   Reason Tests Failed unit tests hadoop.cli.TestHDFSCLI   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.TestGenericRefresh   hadoop.cli.TestAclCLI   hadoop.hdfs.server.namenode.TestFileTruncate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761437/HDFS-9110.02.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c9cb6a5 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12567/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12567/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12567/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12567/console This message was automatically generated.
        Hide
        chelin Charlie Helin added a comment -

        Fixed check style issue

        Show
        chelin Charlie Helin added a comment - Fixed check style issue
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761641/HDFS-9110.03.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 57003fa
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12593/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761641/HDFS-9110.03.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 57003fa Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12593/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 7s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 6s There were no new javac warning messages.
        +1 javadoc 10m 19s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 22s The applied patch generated 3 new checkstyle issues (total was 2, now 4).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 32s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 10s Pre-build of native portion
        -1 hdfs tests 163m 10s Tests failed in hadoop-hdfs.
            209m 12s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestReplaceDatanodeOnFailure
          hadoop.hdfs.web.TestWebHDFSOAuth2
          hadoop.hdfs.server.namenode.ha.TestDNFencing
        Timed out tests org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761650/HDFS-9110.04.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 57003fa
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12596/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12596/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12596/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12596/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 7s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 6s There were no new javac warning messages. +1 javadoc 10m 19s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 22s The applied patch generated 3 new checkstyle issues (total was 2, now 4). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 10s Pre-build of native portion -1 hdfs tests 163m 10s Tests failed in hadoop-hdfs.     209m 12s   Reason Tests Failed unit tests hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.web.TestWebHDFSOAuth2   hadoop.hdfs.server.namenode.ha.TestDNFencing Timed out tests org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761650/HDFS-9110.04.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 57003fa checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12596/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12596/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12596/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12596/console This message was automatically generated.
        Hide
        chelin Charlie Helin added a comment -

        Removed unused imports

        Show
        chelin Charlie Helin added a comment - Removed unused imports
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 5s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 5s There were no new javac warning messages.
        +1 javadoc 10m 23s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 23s There were no new checkstyle issues.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 31s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 16s Pre-build of native portion
        -1 hdfs tests 159m 9s Tests failed in hadoop-hdfs.
            205m 27s  



        Reason Tests
        Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
          org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures
          org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
          org.apache.hadoop.hdfs.server.mover.TestStorageMover



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761691/HDFS-9110.05.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / cc2b473
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12606/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12606/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12606/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 5s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 5s There were no new javac warning messages. +1 javadoc 10m 23s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 23s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 16s Pre-build of native portion -1 hdfs tests 159m 9s Tests failed in hadoop-hdfs.     205m 27s   Reason Tests Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures   org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   org.apache.hadoop.hdfs.server.mover.TestStorageMover Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761691/HDFS-9110.05.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / cc2b473 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12606/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12606/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12606/console This message was automatically generated.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Charlie Helin, please use the "Target Version" field to express your intention. Fix-version is exclusively used by committers when a patch gets committed.

        IAC, 2.6.1 and 2.7.0 are both done. Setting "Target Version" 2.8.0 for this improvement.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Charlie Helin , please use the "Target Version" field to express your intention. Fix-version is exclusively used by committers when a patch gets committed. IAC, 2.6.1 and 2.7.0 are both done. Setting "Target Version" 2.8.0 for this improvement.
        Hide
        chelin Charlie Helin added a comment -

        R vinod kumar got it, sorry!

        Show
        chelin Charlie Helin added a comment - R vinod kumar got it, sorry!
        Hide
        andrew.wang Andrew Wang added a comment -

        Hey Charlie, thanks for working on this, seems like a nice cleanup. Only have some minor comments, overall looks good:

        • Can we avoid reorganizing the imports? This makes backporting harder.
        • I think it'd be good to use the maxDepth variant of walkFileTree, since we're only doing a first-level listing. I think that lets us skip the preVisit step.
        Show
        andrew.wang Andrew Wang added a comment - Hey Charlie, thanks for working on this, seems like a nice cleanup. Only have some minor comments, overall looks good: Can we avoid reorganizing the imports? This makes backporting harder. I think it'd be good to use the maxDepth variant of walkFileTree, since we're only doing a first-level listing. I think that lets us skip the preVisit step.
        Hide
        andrew.wang Andrew Wang added a comment -

        Also I just renamed the JIRA summary to be more descriptive

        Show
        andrew.wang Andrew Wang added a comment - Also I just renamed the JIRA summary to be more descriptive
        Hide
        chelin Charlie Helin added a comment -

        Thanks Andrew Wang!

        Yes it was not expected to change the imports will address also a good idea of setting depth of 1 will incorporate these suggestions.

        Show
        chelin Charlie Helin added a comment - Thanks Andrew Wang ! Yes it was not expected to change the imports will address also a good idea of setting depth of 1 will incorporate these suggestions.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12765615/HDFS-9110.06.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 1107bd3
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12866/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765615/HDFS-9110.06.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1107bd3 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12866/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 20m 48s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 9m 11s There were no new javac warning messages.
        +1 javadoc 11m 45s There were no new javadoc warning messages.
        -1 release audit 0m 22s The applied patch generated 1 release audit warnings.
        -1 checkstyle 1m 40s The applied patch generated 12 new checkstyle issues (total was 2, now 13).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 41s mvn install still works.
        +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse.
        +1 findbugs 2m 54s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 42s Pre-build of native portion
        -1 hdfs tests 230m 37s Tests failed in hadoop-hdfs.
            283m 22s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestEncryptedTransfer
          hadoop.hdfs.server.namenode.TestFSNamesystem
          hadoop.hdfs.TestRollingUpgrade



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12765618/HDFS-9110.07.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 1107bd3
        Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12867/artifact/patchprocess/patchReleaseAuditProblems.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12867/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12867/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12867/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12867/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 20m 48s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 9m 11s There were no new javac warning messages. +1 javadoc 11m 45s There were no new javadoc warning messages. -1 release audit 0m 22s The applied patch generated 1 release audit warnings. -1 checkstyle 1m 40s The applied patch generated 12 new checkstyle issues (total was 2, now 13). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 41s mvn install still works. +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse. +1 findbugs 2m 54s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 42s Pre-build of native portion -1 hdfs tests 230m 37s Tests failed in hadoop-hdfs.     283m 22s   Reason Tests Failed unit tests hadoop.hdfs.TestEncryptedTransfer   hadoop.hdfs.server.namenode.TestFSNamesystem   hadoop.hdfs.TestRollingUpgrade Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765618/HDFS-9110.07.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1107bd3 Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12867/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12867/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12867/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12867/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12867/console This message was automatically generated.
        Hide
        chelin Charlie Helin added a comment -

        Fixed check style issuess

        Show
        chelin Charlie Helin added a comment - Fixed check style issuess
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 20m 32s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 9m 7s There were no new javac warning messages.
        +1 javadoc 11m 43s There were no new javadoc warning messages.
        -1 release audit 0m 21s The applied patch generated 1 release audit warnings.
        +1 checkstyle 1m 31s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 41s mvn install still works.
        +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse.
        +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 4m 44s Pre-build of native portion
        -1 hdfs tests 178m 45s Tests failed in hadoop-hdfs.
            232m 17s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
        Timed out tests org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode
          org.apache.hadoop.hdfs.TestBlockReaderFactory
          org.apache.hadoop.hdfs.server.namenode.TestMalformedURLs



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12765673/HDFS-9110.08.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 8f19538
        Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12891/artifact/patchprocess/patchReleaseAuditProblems.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12891/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12891/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12891/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 20m 32s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 9m 7s There were no new javac warning messages. +1 javadoc 11m 43s There were no new javadoc warning messages. -1 release audit 0m 21s The applied patch generated 1 release audit warnings. +1 checkstyle 1m 31s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 41s mvn install still works. +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse. +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 4m 44s Pre-build of native portion -1 hdfs tests 178m 45s Tests failed in hadoop-hdfs.     232m 17s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints Timed out tests org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode   org.apache.hadoop.hdfs.TestBlockReaderFactory   org.apache.hadoop.hdfs.server.namenode.TestMalformedURLs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765673/HDFS-9110.08.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8f19538 Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12891/artifact/patchprocess/patchReleaseAuditProblems.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12891/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12891/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12891/console This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Tests look like flakes and the RAT check is the standing tree.h issue, +1 LGTM will commit. Thanks Charlie!

        Show
        andrew.wang Andrew Wang added a comment - Tests look like flakes and the RAT check is the standing tree.h issue, +1 LGTM will commit. Thanks Charlie!
        Hide
        andrew.wang Andrew Wang added a comment -

        Committed to trunk and branch-2, thx again Charlie for the contribution!

        Show
        andrew.wang Andrew Wang added a comment - Committed to trunk and branch-2, thx again Charlie for the contribution!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8602 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8602/)
        HDFS-9110. Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8602 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8602/ ) HDFS-9110 . Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #516 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/516/)
        HDFS-9110. Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #516 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/516/ ) HDFS-9110 . Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/477/)
        HDFS-9110. Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/477/ ) HDFS-9110 . Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #505 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/505/)
        HDFS-9110. Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #505 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/505/ ) HDFS-9110 . Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2450 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2450/)
        HDFS-9110. Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2450 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2450/ ) HDFS-9110 . Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #1243 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1243/)
        HDFS-9110. Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1243 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1243/ ) HDFS-9110 . Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2416 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2416/)
        HDFS-9110. Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2416 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2416/ ) HDFS-9110 . Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for (wang: rev 357b1fd0822447f9e73a20c69f37006d9a37ecbc) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

          • Assignee:
            chelin Charlie Helin
            Reporter:
            chelin Charlie Helin
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development