Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8865

Improve quota initialization performance

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1, 2.6.6, 2.7.5
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      After replaying edits, the whole file system tree is recursively scanned in order to initialize the quota. For big name space, this can take a very long time. Since this is done during namenode failover, it also affects failover latency.

      By using the Fork-Join framework, I was able to greatly reduce the initialization time. The following is the test result using the fsimage from one of the big name nodes we have.

      threads seconds
      1 (existing) 55
      1 (fork-join) 68
      4 16
      8 8
      12 6
      16 5
      20 4
      1. HDFS-8865.patch
        18 kB
        Kihwal Lee
      2. HDFS-8865.v2.patch
        18 kB
        Kihwal Lee
      3. HDFS-8865.v2.checkstyle.patch
        18 kB
        Kihwal Lee
      4. HDFS-8865.v3.patch
        18 kB
        Kihwal Lee
      5. HDFS-8865_branch-2.7.patch
        18 kB
        Kihwal Lee
      6. HDFS-8865_branch-2.6.patch
        11 kB
        Nicolas Fraison
      7. HDFS-8865.branch-2.6.patch
        21 kB
        Xiao Chen
      8. HDFS-8865.branch-2.7.patch
        19 kB
        Xiao Chen
      9. HDFS-8865.branch-2.6.01.patch
        15 kB
        Xiao Chen

        Issue Links

          Activity

          Hide
          xiaochen Xiao Chen added a comment -

          Thanks a lot Yongjun Zhang for review. Pushed to branch-2.7 and branch-2.6. Compiled again before pushing.
          Also pushed the said series of jira to fix branch-2.6 test, and filed HDFS-12578 for branch-2.7.

          Thanks Kihwal Lee Nicolas Fraison for contributing the patches, and Wei-Chiu Chuang for initial backport/review.

          Show
          xiaochen Xiao Chen added a comment - Thanks a lot Yongjun Zhang for review. Pushed to branch-2.7 and branch-2.6. Compiled again before pushing. Also pushed the said series of jira to fix branch-2.6 test, and filed HDFS-12578 for branch-2.7. Thanks Kihwal Lee Nicolas Fraison for contributing the patches, and Wei-Chiu Chuang for initial backport/review.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks Xiao Chen, +1 on the updated 2.6 patch (01).

          Show
          yzhangal Yongjun Zhang added a comment - Thanks Xiao Chen , +1 on the updated 2.6 patch (01).
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Yongjun Zhang, attached new 2.6 patch.

          Show
          xiaochen Xiao Chen added a comment - Thanks Yongjun Zhang , attached new 2.6 patch.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks Xiao Chen for working on this.

          +1 on the new revs for 2.7.

          Question on 2.6: Good to add the test to the 2.6 new patch. There is only one new test for this jira in 2.7 and beyond. Though it helps testing by including additional tests, should the other tests come with backporting different jiras (HDFS-10843 etc)?

          Minor: when uploading new patches for the same branch in the future, suggest to include a version string.

          Show
          yzhangal Yongjun Zhang added a comment - Thanks Xiao Chen for working on this. +1 on the new revs for 2.7. Question on 2.6: Good to add the test to the 2.6 new patch. There is only one new test for this jira in 2.7 and beyond. Though it helps testing by including additional tests, should the other tests come with backporting different jiras ( HDFS-10843 etc)? Minor: when uploading new patches for the same branch in the future, suggest to include a version string.
          Hide
          xiaochen Xiao Chen added a comment -

          I gave up on pre-commits and went on to run hadoop-hdfs tests in internal infra. Results below:

          branch-2.7:
          Test Result (12 failures / +12)
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten
          org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling
          org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode.testNonDFSUsedONDeadNodeReReg
          org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testSnapshotWithAppendTruncate
          org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testCopyOnTruncateWithDataNodesRestart
          org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testSnapshotWithTruncates
          org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testTruncateRecovery
          org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testTruncateShellCommandOnBlockBoundary
          org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testTruncateWithDataNodesRestartImmediately
          org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testTruncate4Symlink
          
          branch-2.6:
          Test Result (7 failures / -5)
          org.apache.hadoop.hdfs.server.datanode.TestBPOfferService.testBasicFunctionality
          org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner.testDataBlockScanner
          org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner.testBlockScannerAfterRefresh
          org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner.testBlockScannerAfterRestart
          org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles.testFallbackToDiskFull
          org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache.testDataXceiverCleansUpSlotsOnFailure
          

          All passed locally, except TestDeadDatanode#testNonDFSUsedONDeadNodeReReg for branch-2.7 and TestLazyPersistFiles#testFallbackToDiskFull for branch-2.6 are failing with or without this patch.
          Will file a jira for branch-2.7, and backport the following (in the exact order, left is earlier) HDFS-9072>HDFS-9740>HDFS-10270->HDFS-9743 to fix branch-2.6.

          Plan to commit this by end of Monday if no objections.
          Thanks.

          Show
          xiaochen Xiao Chen added a comment - I gave up on pre-commits and went on to run hadoop-hdfs tests in internal infra. Results below: branch-2.7: Test Result (12 failures / +12) org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode.testNonDFSUsedONDeadNodeReReg org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testSnapshotWithAppendTruncate org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testCopyOnTruncateWithDataNodesRestart org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testSnapshotWithTruncates org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testTruncateRecovery org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testTruncateShellCommandOnBlockBoundary org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testTruncateWithDataNodesRestartImmediately org.apache.hadoop.hdfs.server.namenode.TestFileTruncate.testTruncate4Symlink branch-2.6: Test Result (7 failures / -5) org.apache.hadoop.hdfs.server.datanode.TestBPOfferService.testBasicFunctionality org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner.testDataBlockScanner org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner.testBlockScannerAfterRefresh org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner.testBlockScannerAfterRestart org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles.testFallbackToDiskFull org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache.testDataXceiverCleansUpSlotsOnFailure All passed locally, except TestDeadDatanode#testNonDFSUsedONDeadNodeReReg for branch-2.7 and TestLazyPersistFiles#testFallbackToDiskFull for branch-2.6 are failing with or without this patch. Will file a jira for branch-2.7, and backport the following (in the exact order, left is earlier) HDFS-9072 > HDFS-9740 > HDFS-10270 -> HDFS-9743 to fix branch-2.6. Plan to commit this by end of Monday if no objections. Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 24s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
                branch-2.7 Compile Tests
          +1 mvninstall 6m 44s branch-2.7 passed
          +1 compile 1m 13s branch-2.7 passed with JDK v1.8.0_144
          +1 compile 1m 8s branch-2.7 passed with JDK v1.7.0_151
          +1 checkstyle 0m 32s branch-2.7 passed
          +1 mvnsite 1m 5s branch-2.7 passed
          +1 findbugs 3m 3s branch-2.7 passed
          +1 javadoc 1m 11s branch-2.7 passed with JDK v1.8.0_144
          +1 javadoc 1m 51s branch-2.7 passed with JDK v1.7.0_151
                Patch Compile Tests
          +1 mvninstall 0m 52s the patch passed
          +1 compile 0m 58s the patch passed with JDK v1.8.0_144
          +1 javac 0m 58s the patch passed
          +1 compile 0m 59s the patch passed with JDK v1.7.0_151
          +1 javac 0m 59s the patch passed
          -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 612 unchanged - 3 fixed = 613 total (was 615)
          +1 mvnsite 0m 56s the patch passed
          -1 whitespace 0m 0s The patch has 60 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 3m 7s the patch passed
          +1 javadoc 1m 2s the patch passed with JDK v1.8.0_144
          +1 javadoc 1m 55s the patch passed with JDK v1.7.0_151
                Other Tests
          -1 unit 683m 58s hadoop-hdfs in the patch failed with JDK v1.7.0_151.
          -1 asflicense 13m 41s The patch generated 105 ASF License warnings.
          783m 16s



          Reason Tests
          JDK v1.8.0_144 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.TestDeadDatanode
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
          JDK v1.8.0_144 Timed out junit tests org.apache.hadoop.hdfs.web.TestWebHdfsTokens
          JDK v1.7.0_151 Failed junit tests hadoop.hdfs.server.namenode.TestSaveNamespace
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotManager
            hadoop.hdfs.server.namenode.TestMetadataVersionOutput
            hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
            hadoop.hdfs.TestMissingBlocksAlert
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
            hadoop.hdfs.server.namenode.TestNameNodeOptionParsing
            hadoop.hdfs.server.namenode.TestGetBlockLocations
            hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
            hadoop.hdfs.server.namenode.TestCreateEditsLog
          JDK v1.7.0_151 Timed out junit tests org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
            org.apache.hadoop.hdfs.TestEncryptionZones
            org.apache.hadoop.hdfs.TestLeaseRecovery2
            org.apache.hadoop.hdfs.server.namenode.TestStartup
            org.apache.hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
            org.apache.hadoop.hdfs.tools.TestDFSAdminWithHA
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
            org.apache.hadoop.hdfs.TestFileCreationEmpty
            org.apache.hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy
            org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
            org.apache.hadoop.hdfs.qjournal.client.TestQuorumJournalManager
            org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
            org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
            org.apache.hadoop.hdfs.TestDatanodeRegistration
            org.apache.hadoop.hdfs.server.blockmanagement.TestBlockManager
            org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServer
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
            org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl
            org.apache.hadoop.hdfs.TestDFSClientFailover
            org.apache.hadoop.hdfs.TestSetrepDecreasing
            org.apache.hadoop.hdfs.TestReservedRawPaths
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
            org.apache.hadoop.hdfs.server.namenode.TestAddBlockRetry
            org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
            org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus
            org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
            org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
            org.apache.hadoop.hdfs.server.namenode.TestLeaseManager
            org.apache.hadoop.hdfs.qjournal.server.TestJournalNode
            org.apache.hadoop.hdfs.TestQuota
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            org.apache.hadoop.hdfs.server.namenode.TestINodeFile
            org.apache.hadoop.hdfs.server.namenode.TestEditLog
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl
            org.apache.hadoop.hdfs.TestDataTransferKeepalive
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
            org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            org.apache.hadoop.hdfs.TestDatanodeDeath
            org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings
            org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
            org.apache.hadoop.hdfs.TestFileAppend
            org.apache.hadoop.hdfs.TestPread
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
            org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            org.apache.hadoop.hdfs.server.namenode.TestBackupNode
            org.apache.hadoop.hdfs.server.namenode.TestFSImage
            org.apache.hadoop.hdfs.TestSafeMode
            org.apache.hadoop.hdfs.server.namenode.TestQuotaByStorageType
            org.apache.hadoop.hdfs.TestFsShellPermission
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
            org.apache.hadoop.hdfs.TestFileAppend4
            org.apache.hadoop.hdfs.TestDFSFinalize
            org.apache.hadoop.hdfs.server.namenode.TestStreamFile
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys
            org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
            org.apache.hadoop.hdfs.TestRead
            org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults
            org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
            org.apache.hadoop.hdfs.server.namenode.TestFileContextXAttr
            org.apache.hadoop.hdfs.server.namenode.TestFSImageWithXAttr
            org.apache.hadoop.hdfs.TestDFSPermission
            org.apache.hadoop.hdfs.web.TestWebHdfsTokens
            org.apache.hadoop.hdfs.server.namenode.TestFSNamesystemMBean
            org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives
            org.apache.hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs
            org.apache.hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
            org.apache.hadoop.hdfs.security.TestDelegationToken
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeXAttr
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
            org.apache.hadoop.hdfs.server.namenode.TestFileLimit
            org.apache.hadoop.hdfs.TestFileCorruption
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            org.apache.hadoop.hdfs.server.namenode.TestFsck
            org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
            org.apache.hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
            org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing
            org.apache.hadoop.hdfs.TestCrcCorruption
            org.apache.hadoop.hdfs.TestFileCreationDelete
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
            org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
            org.apache.hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
            org.apache.hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
            org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM
            org.apache.hadoop.hdfs.TestBlockStoragePolicy
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename
            org.apache.hadoop.hdfs.server.namenode.TestFileTruncate
            org.apache.hadoop.hdfs.TestRollingUpgrade
            org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
            org.apache.hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
            org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
            org.apache.hadoop.hdfs.server.namenode.TestTruncateQuotaUpdate
            org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode
            org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures
            org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
            org.apache.hadoop.hdfs.TestDFSMkdirs
            org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
            org.apache.hadoop.hdfs.TestLease
            org.apache.hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes
            org.apache.hadoop.hdfs.server.namenode.TestMetaSave
            org.apache.hadoop.hdfs.qjournal.TestNNWithQJM
            org.apache.hadoop.hdfs.web.TestWebHDFS
            org.apache.hadoop.hdfs.server.namenode.TestCommitBlockWithInvalidGenStamp
            org.apache.hadoop.hdfs.server.namenode.TestINodeAttributeProvider
            org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
            org.apache.hadoop.hdfs.tools.TestDFSHAAdminMiniCluster
            org.apache.hadoop.hdfs.tools.TestGetGroups
            org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount
            org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
            org.apache.hadoop.hdfs.web.TestWebHDFSXAttr
            org.apache.hadoop.hdfs.TestRollingUpgradeRollback
            org.apache.hadoop.hdfs.TestBlockReaderLocal
            org.apache.hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController
            org.apache.hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes
            org.apache.hadoop.hdfs.server.namenode.TestFSDirectory
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication
            org.apache.hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA
            org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
            org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup
            org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
            org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader
            org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache
            org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication
            org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl
            org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
            org.apache.hadoop.hdfs.server.namenode.TestXAttrConfigFlag
            org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
            org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat
            org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover
            org.apache.hadoop.hdfs.server.namenode.TestDeleteRace
            org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
            org.apache.hadoop.hdfs.server.namenode.TestAddBlock
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
            org.apache.hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA
            org.apache.hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
            org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
            org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
            org.apache.hadoop.hdfs.TestBlockReaderFactory
            org.apache.hadoop.hdfs.TestDistributedFileSystem
            org.apache.hadoop.hdfs.server.blockmanagement.TestSequentialBlockId
            org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            org.apache.hadoop.hdfs.web.TestWebHDFSForHA
            org.apache.hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
            org.apache.hadoop.hdfs.TestEncryptedTransfer
            org.apache.hadoop.hdfs.TestDFSClientExcludedNodes
            org.apache.hadoop.hdfs.server.namenode.TestAuditLogs
            org.apache.hadoop.hdfs.TestDFSShell
            org.apache.hadoop.hdfs.server.namenode.TestHostsFiles
            org.apache.hadoop.hdfs.server.namenode.TestAuditLogger
            org.apache.hadoop.hdfs.TestPersistBlocks
            org.apache.hadoop.hdfs.tools.TestDebugAdmin
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
            org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode
            org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
            org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
            org.apache.hadoop.hdfs.TestGetFileChecksum
            org.apache.hadoop.hdfs.web.TestWebHDFSAcl
            org.apache.hadoop.hdfs.TestAbandonBlock



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:67e87c9
          JIRA Issue HDFS-8865
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12889763/HDFS-8865.branch-2.7.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
          uname Linux e18536e63d2c 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 90aae39
          Default Java 1.7.0_151
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_144 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_151
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21463/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/21463/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/21463/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_151.txt
          JDK v1.7.0_151 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21463/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/21463/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21463/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.       branch-2.7 Compile Tests +1 mvninstall 6m 44s branch-2.7 passed +1 compile 1m 13s branch-2.7 passed with JDK v1.8.0_144 +1 compile 1m 8s branch-2.7 passed with JDK v1.7.0_151 +1 checkstyle 0m 32s branch-2.7 passed +1 mvnsite 1m 5s branch-2.7 passed +1 findbugs 3m 3s branch-2.7 passed +1 javadoc 1m 11s branch-2.7 passed with JDK v1.8.0_144 +1 javadoc 1m 51s branch-2.7 passed with JDK v1.7.0_151       Patch Compile Tests +1 mvninstall 0m 52s the patch passed +1 compile 0m 58s the patch passed with JDK v1.8.0_144 +1 javac 0m 58s the patch passed +1 compile 0m 59s the patch passed with JDK v1.7.0_151 +1 javac 0m 59s the patch passed -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 612 unchanged - 3 fixed = 613 total (was 615) +1 mvnsite 0m 56s the patch passed -1 whitespace 0m 0s The patch has 60 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 3m 7s the patch passed +1 javadoc 1m 2s the patch passed with JDK v1.8.0_144 +1 javadoc 1m 55s the patch passed with JDK v1.7.0_151       Other Tests -1 unit 683m 58s hadoop-hdfs in the patch failed with JDK v1.7.0_151. -1 asflicense 13m 41s The patch generated 105 ASF License warnings. 783m 16s Reason Tests JDK v1.8.0_144 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.TestDeadDatanode   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes JDK v1.8.0_144 Timed out junit tests org.apache.hadoop.hdfs.web.TestWebHdfsTokens JDK v1.7.0_151 Failed junit tests hadoop.hdfs.server.namenode.TestSaveNamespace   hadoop.hdfs.server.namenode.snapshot.TestSnapshotManager   hadoop.hdfs.server.namenode.TestMetadataVersionOutput   hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork   hadoop.hdfs.TestMissingBlocksAlert   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.namenode.TestSecurityTokenEditLog   hadoop.hdfs.server.namenode.TestNameNodeOptionParsing   hadoop.hdfs.server.namenode.TestGetBlockLocations   hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement   hadoop.hdfs.server.namenode.TestCreateEditsLog JDK v1.7.0_151 Timed out junit tests org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend   org.apache.hadoop.hdfs.TestEncryptionZones   org.apache.hadoop.hdfs.TestLeaseRecovery2   org.apache.hadoop.hdfs.server.namenode.TestStartup   org.apache.hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   org.apache.hadoop.hdfs.tools.TestDFSAdminWithHA   org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing   org.apache.hadoop.hdfs.TestFileCreationEmpty   org.apache.hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy   org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade   org.apache.hadoop.hdfs.qjournal.client.TestQuorumJournalManager   org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer   org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   org.apache.hadoop.hdfs.TestDatanodeRegistration   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockManager   org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServer   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport   org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl   org.apache.hadoop.hdfs.TestDFSClientFailover   org.apache.hadoop.hdfs.TestSetrepDecreasing   org.apache.hadoop.hdfs.TestReservedRawPaths   org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   org.apache.hadoop.hdfs.server.namenode.TestAddBlockRetry   org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure   org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus   org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   org.apache.hadoop.hdfs.server.namenode.TestStorageRestore   org.apache.hadoop.hdfs.server.namenode.TestLeaseManager   org.apache.hadoop.hdfs.qjournal.server.TestJournalNode   org.apache.hadoop.hdfs.TestQuota   org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   org.apache.hadoop.hdfs.server.namenode.TestINodeFile   org.apache.hadoop.hdfs.server.namenode.TestEditLog   org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl   org.apache.hadoop.hdfs.TestDataTransferKeepalive   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot   org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer   org.apache.hadoop.hdfs.TestDatanodeDeath   org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings   org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart   org.apache.hadoop.hdfs.TestFileAppend   org.apache.hadoop.hdfs.TestPread   org.apache.hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot   org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   org.apache.hadoop.hdfs.server.namenode.TestBackupNode   org.apache.hadoop.hdfs.server.namenode.TestFSImage   org.apache.hadoop.hdfs.TestSafeMode   org.apache.hadoop.hdfs.server.namenode.TestQuotaByStorageType   org.apache.hadoop.hdfs.TestFsShellPermission   org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot   org.apache.hadoop.hdfs.TestFileAppend4   org.apache.hadoop.hdfs.TestDFSFinalize   org.apache.hadoop.hdfs.server.namenode.TestStreamFile   org.apache.hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys   org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   org.apache.hadoop.hdfs.TestRead   org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults   org.apache.hadoop.hdfs.TestDFSUpgradeFromImage   org.apache.hadoop.hdfs.server.namenode.TestFileContextXAttr   org.apache.hadoop.hdfs.server.namenode.TestFSImageWithXAttr   org.apache.hadoop.hdfs.TestDFSPermission   org.apache.hadoop.hdfs.web.TestWebHdfsTokens   org.apache.hadoop.hdfs.server.namenode.TestFSNamesystemMBean   org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives   org.apache.hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs   org.apache.hadoop.hdfs.server.namenode.TestFSImageWithSnapshot   org.apache.hadoop.hdfs.security.TestDelegationToken   org.apache.hadoop.hdfs.server.namenode.TestNameNodeXAttr   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap   org.apache.hadoop.hdfs.server.namenode.TestFileLimit   org.apache.hadoop.hdfs.TestFileCorruption   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot   org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   org.apache.hadoop.hdfs.server.namenode.TestFsck   org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled   org.apache.hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing   org.apache.hadoop.hdfs.TestCrcCorruption   org.apache.hadoop.hdfs.TestFileCreationDelete   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters   org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS   org.apache.hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer   org.apache.hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM   org.apache.hadoop.hdfs.TestBlockStoragePolicy   org.apache.hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename   org.apache.hadoop.hdfs.server.namenode.TestFileTruncate   org.apache.hadoop.hdfs.TestRollingUpgrade   org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality   org.apache.hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate   org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks   org.apache.hadoop.hdfs.server.namenode.TestTruncateQuotaUpdate   org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode   org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures   org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks   org.apache.hadoop.hdfs.TestDFSMkdirs   org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite   org.apache.hadoop.hdfs.TestLease   org.apache.hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes   org.apache.hadoop.hdfs.server.namenode.TestMetaSave   org.apache.hadoop.hdfs.qjournal.TestNNWithQJM   org.apache.hadoop.hdfs.web.TestWebHDFS   org.apache.hadoop.hdfs.server.namenode.TestCommitBlockWithInvalidGenStamp   org.apache.hadoop.hdfs.server.namenode.TestINodeAttributeProvider   org.apache.hadoop.hdfs.server.namenode.TestCheckpoint   org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean   org.apache.hadoop.hdfs.tools.TestDFSHAAdminMiniCluster   org.apache.hadoop.hdfs.tools.TestGetGroups   org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount   org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs   org.apache.hadoop.hdfs.web.TestWebHDFSXAttr   org.apache.hadoop.hdfs.TestRollingUpgradeRollback   org.apache.hadoop.hdfs.TestBlockReaderLocal   org.apache.hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController   org.apache.hadoop.hdfs.server.namenode.TestNamenodeRetryCache   org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes   org.apache.hadoop.hdfs.server.namenode.TestFSDirectory   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication   org.apache.hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA   org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup   org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication   org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader   org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache   org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication   org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl   org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   org.apache.hadoop.hdfs.server.namenode.TestXAttrConfigFlag   org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead   org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat   org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover   org.apache.hadoop.hdfs.server.namenode.TestDeleteRace   org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   org.apache.hadoop.hdfs.server.namenode.TestEditLogRace   org.apache.hadoop.hdfs.server.namenode.TestAddBlock   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   org.apache.hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA   org.apache.hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd   org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs   org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction   org.apache.hadoop.hdfs.TestBlockReaderFactory   org.apache.hadoop.hdfs.TestDistributedFileSystem   org.apache.hadoop.hdfs.server.blockmanagement.TestSequentialBlockId   org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   org.apache.hadoop.hdfs.web.TestWebHDFSForHA   org.apache.hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   org.apache.hadoop.hdfs.TestEncryptedTransfer   org.apache.hadoop.hdfs.TestDFSClientExcludedNodes   org.apache.hadoop.hdfs.server.namenode.TestAuditLogs   org.apache.hadoop.hdfs.TestDFSShell   org.apache.hadoop.hdfs.server.namenode.TestHostsFiles   org.apache.hadoop.hdfs.server.namenode.TestAuditLogger   org.apache.hadoop.hdfs.TestPersistBlocks   org.apache.hadoop.hdfs.tools.TestDebugAdmin   org.apache.hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics   org.apache.hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot   org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode   org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens   org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics   org.apache.hadoop.hdfs.TestGetFileChecksum   org.apache.hadoop.hdfs.web.TestWebHDFSAcl   org.apache.hadoop.hdfs.TestAbandonBlock Subsystem Report/Notes Docker Image:yetus/hadoop:67e87c9 JIRA Issue HDFS-8865 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12889763/HDFS-8865.branch-2.7.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml uname Linux e18536e63d2c 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 90aae39 Default Java 1.7.0_151 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_144 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_151 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21463/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/21463/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/21463/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_151.txt JDK v1.7.0_151 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21463/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/21463/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21463/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 11m 19s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
                branch-2.7 Compile Tests
          +1 mvninstall 7m 49s branch-2.7 passed
          +1 compile 0m 59s branch-2.7 passed with JDK v1.8.0_144
          +1 compile 0m 59s branch-2.7 passed with JDK v1.7.0_151
          +1 checkstyle 0m 31s branch-2.7 passed
          +1 mvnsite 1m 0s branch-2.7 passed
          +1 findbugs 2m 55s branch-2.7 passed
          +1 javadoc 0m 57s branch-2.7 passed with JDK v1.8.0_144
          +1 javadoc 1m 43s branch-2.7 passed with JDK v1.7.0_151
                Patch Compile Tests
          +1 mvninstall 0m 52s the patch passed
          +1 compile 0m 56s the patch passed with JDK v1.8.0_144
          +1 javac 0m 56s the patch passed
          +1 compile 0m 57s the patch passed with JDK v1.7.0_151
          +1 javac 0m 57s the patch passed
          -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 611 unchanged - 3 fixed = 612 total (was 614)
          +1 mvnsite 0m 56s the patch passed
          -1 whitespace 0m 0s The patch has 60 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 3m 3s the patch passed
          +1 javadoc 0m 55s the patch passed with JDK v1.8.0_144
          +1 javadoc 1m 49s the patch passed with JDK v1.7.0_151
                Other Tests
          -1 unit 696m 51s hadoop-hdfs in the patch failed with JDK v1.7.0_151.
          -1 asflicense 12m 48s The patch generated 57 ASF License warnings.
          801m 51s



          Reason Tests
          JDK v1.8.0_144 Failed junit tests hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.namenode.TestDeadDatanode
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.web.TestWebHdfsTokens
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          JDK v1.7.0_151 Failed junit tests hadoop.hdfs.server.namenode.TestLeaseManager
            hadoop.hdfs.server.datanode.TestDataNodeInitStorage
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.server.namenode.TestNameNodeOptionParsing
            hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
            hadoop.hdfs.tools.TestDFSAdminWithHA
            hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.TestFileLengthOnClusterRestart
            hadoop.hdfs.server.datanode.TestBPOfferService
            hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
            hadoop.hdfs.server.datanode.fsdataset.TestAvailableSpaceVolumeChoosingPolicy
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.hdfs.server.namenode.ha.TestDNFencing
          JDK v1.7.0_151 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestHSync
            org.apache.hadoop.hdfs.TestModTime
            org.apache.hadoop.hdfs.TestEncryptionZones
            org.apache.hadoop.hdfs.TestLeaseRecovery2
            org.apache.hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
            org.apache.hadoop.hdfs.server.namenode.TestAllowFormat
            org.apache.hadoop.hdfs.TestParallelShortCircuitRead
            org.apache.hadoop.hdfs.TestDFSStartupVersions
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool
            org.apache.hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage
            org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling
            org.apache.hadoop.hdfs.TestFileCreationEmpty
            org.apache.hadoop.hdfs.TestFileCreationClient
            org.apache.hadoop.hdfs.qjournal.client.TestQuorumJournalManager
            org.apache.hadoop.hdfs.TestListFilesInFileContext
            org.apache.hadoop.hdfs.TestDatanodeRegistration
            org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServer
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            org.apache.hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport
            org.apache.hadoop.hdfs.server.blockmanagement.TestBlockManager
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
            org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl
            org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
            org.apache.hadoop.hdfs.TestBlocksScheduledCounter
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeExit
            org.apache.hadoop.hdfs.TestDFSClientFailover
            org.apache.hadoop.hdfs.TestMultiThreadedHflush
            org.apache.hadoop.hdfs.TestSetrepDecreasing
            org.apache.hadoop.hdfs.TestReservedRawPaths
            org.apache.hadoop.hdfs.TestSetrepIncreasing
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery
            org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement
            org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
            org.apache.hadoop.hdfs.server.datanode.TestReadOnlySharedStorage
            org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes
            org.apache.hadoop.hdfs.TestFileCreation
            org.apache.hadoop.hdfs.TestQuota
            org.apache.hadoop.hdfs.qjournal.server.TestJournalNode
            org.apache.hadoop.hdfs.server.namenode.TestINodeFile
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl
            org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery
            org.apache.hadoop.hdfs.TestReplication
            org.apache.hadoop.hdfs.TestDataTransferKeepalive
            org.apache.hadoop.hdfs.TestDatanodeDeath
            org.apache.hadoop.hdfs.server.datanode.TestTriggerBlockReport
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            org.apache.hadoop.hdfs.TestPread
            org.apache.hadoop.hdfs.TestFileAppend
            org.apache.hadoop.hdfs.TestSafeMode
            org.apache.hadoop.hdfs.TestDFSClientRetries
            org.apache.hadoop.hdfs.TestBlockMissingException
            org.apache.hadoop.hdfs.TestDFSFinalize
            org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
            org.apache.hadoop.hdfs.TestFileAppend4
            org.apache.hadoop.hdfs.server.namenode.TestStreamFile
            org.apache.hadoop.hdfs.TestFileAppend3
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys
            org.apache.hadoop.hdfs.TestFileAppend2
            org.apache.hadoop.hdfs.TestRead
            org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults
            org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
            org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade
            org.apache.hadoop.hdfs.TestHDFSFileSystemContract
            org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
            org.apache.hadoop.hdfs.TestDFSPermission
            org.apache.hadoop.hdfs.web.TestWebHdfsTokens
            org.apache.hadoop.hdfs.server.datanode.TestBatchIbr
            org.apache.hadoop.hdfs.TestDecommission
            org.apache.hadoop.hdfs.TestReadWhileWriting
            org.apache.hadoop.hdfs.security.TestDelegationToken
            org.apache.hadoop.hdfs.TestFileAppendRestart
            org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
            org.apache.hadoop.hdfs.TestFileCorruption
            org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
            org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
            org.apache.hadoop.hdfs.TestCrcCorruption
            org.apache.hadoop.hdfs.TestFileCreationDelete
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
            org.apache.hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
            org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
            org.apache.hadoop.hdfs.TestDFSAddressConfig
            org.apache.hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
            org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM
            org.apache.hadoop.hdfs.TestLeaseRecovery
            org.apache.hadoop.hdfs.TestBlockStoragePolicy
            org.apache.hadoop.hdfs.TestRollingUpgrade
            org.apache.hadoop.hdfs.TestDFSRemove
            org.apache.hadoop.hdfs.TestSeekBug
            org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
            org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
            org.apache.hadoop.hdfs.TestLocalDFS
            org.apache.hadoop.hdfs.TestDFSMkdirs
            org.apache.hadoop.hdfs.TestHDFSServerPorts
            org.apache.hadoop.hdfs.TestLease
            org.apache.hadoop.hdfs.TestDFSUpgrade
            org.apache.hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes
            org.apache.hadoop.hdfs.server.namenode.TestMetaSave
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
            org.apache.hadoop.hdfs.qjournal.TestNNWithQJM
            org.apache.hadoop.hdfs.web.TestWebHDFS
            org.apache.hadoop.hdfs.TestAppendSnapshotTruncate
            org.apache.hadoop.hdfs.tools.TestGetGroups
            org.apache.hadoop.hdfs.tools.TestDFSHAAdminMiniCluster
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestRbwSpaceReservation
            org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount
            org.apache.hadoop.hdfs.TestBlockReaderLocal
            org.apache.hadoop.hdfs.web.TestWebHDFSXAttr
            org.apache.hadoop.hdfs.TestRollingUpgradeRollback
            org.apache.hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            org.apache.hadoop.hdfs.server.datanode.TestDiskError
            org.apache.hadoop.hdfs.TestRestartDFS
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
            org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes
            org.apache.hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold
            org.apache.hadoop.hdfs.server.datanode.TestIncrementalBlockReports
            org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
            org.apache.hadoop.hdfs.server.datanode.TestIncrementalBrVariations
            org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache
            org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
            org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup
            org.apache.hadoop.hdfs.TestRenameWhileOpen
            org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache
            org.apache.hadoop.hdfs.server.datanode.TestTransferRbw
            org.apache.hadoop.hdfs.TestFileConcurrentReader
            org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
            org.apache.hadoop.hdfs.server.namenode.TestXAttrConfigFlag
            org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
            org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication
            org.apache.hadoop.hdfs.TestFSOutputSummer
            org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
            org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover
            org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation
            org.apache.hadoop.hdfs.TestDFSRollback
            org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            org.apache.hadoop.hdfs.TestMiniDFSCluster
            org.apache.hadoop.hdfs.server.datanode.TestBlockScanner
            org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
            org.apache.hadoop.hdfs.TestBlockReaderFactory
            org.apache.hadoop.hdfs.qjournal.client.TestEpochsAreUnique
            org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy
            org.apache.hadoop.hdfs.TestDistributedFileSystem
            org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            org.apache.hadoop.hdfs.server.blockmanagement.TestSequentialBlockId
            org.apache.hadoop.hdfs.TestAppendDifferentChecksum
            org.apache.hadoop.hdfs.TestHFlush
            org.apache.hadoop.hdfs.web.TestWebHDFSForHA
            org.apache.hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
            org.apache.hadoop.hdfs.TestBalancerBandwidth
            org.apache.hadoop.hdfs.TestClientReportBadBlock
            org.apache.hadoop.hdfs.TestDFSClientExcludedNodes
            org.apache.hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages
            org.apache.hadoop.hdfs.TestSetTimes
            org.apache.hadoop.hdfs.TestDFSShell
            org.apache.hadoop.hdfs.server.namenode.TestHostsFiles
            org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
            org.apache.hadoop.hdfs.server.datanode.TestCachingStrategy
            org.apache.hadoop.hdfs.tools.TestDebugAdmin
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
            org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode
            org.apache.hadoop.hdfs.TestDFSRename
            org.apache.hadoop.hdfs.tools.TestDFSAdmin
            org.apache.hadoop.hdfs.TestGetBlocks
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            org.apache.hadoop.hdfs.TestGetFileChecksum
            org.apache.hadoop.hdfs.TestAbandonBlock
            org.apache.hadoop.hdfs.TestDFSShellGenericOptions
            org.apache.hadoop.hdfs.web.TestWebHDFSAcl



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:67e87c9
          JIRA Issue HDFS-8865
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12889763/HDFS-8865.branch-2.7.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
          uname Linux e3f1f9830046 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 90aae39
          Default Java 1.7.0_151
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_144 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_151
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21443/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/21443/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/21443/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_151.txt
          JDK v1.7.0_151 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21443/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/21443/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21443/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 19s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.       branch-2.7 Compile Tests +1 mvninstall 7m 49s branch-2.7 passed +1 compile 0m 59s branch-2.7 passed with JDK v1.8.0_144 +1 compile 0m 59s branch-2.7 passed with JDK v1.7.0_151 +1 checkstyle 0m 31s branch-2.7 passed +1 mvnsite 1m 0s branch-2.7 passed +1 findbugs 2m 55s branch-2.7 passed +1 javadoc 0m 57s branch-2.7 passed with JDK v1.8.0_144 +1 javadoc 1m 43s branch-2.7 passed with JDK v1.7.0_151       Patch Compile Tests +1 mvninstall 0m 52s the patch passed +1 compile 0m 56s the patch passed with JDK v1.8.0_144 +1 javac 0m 56s the patch passed +1 compile 0m 57s the patch passed with JDK v1.7.0_151 +1 javac 0m 57s the patch passed -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 611 unchanged - 3 fixed = 612 total (was 614) +1 mvnsite 0m 56s the patch passed -1 whitespace 0m 0s The patch has 60 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 3m 3s the patch passed +1 javadoc 0m 55s the patch passed with JDK v1.8.0_144 +1 javadoc 1m 49s the patch passed with JDK v1.7.0_151       Other Tests -1 unit 696m 51s hadoop-hdfs in the patch failed with JDK v1.7.0_151. -1 asflicense 12m 48s The patch generated 57 ASF License warnings. 801m 51s Reason Tests JDK v1.8.0_144 Failed junit tests hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.TestDeadDatanode   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots JDK v1.7.0_151 Failed junit tests hadoop.hdfs.server.namenode.TestLeaseManager   hadoop.hdfs.server.datanode.TestDataNodeInitStorage   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.namenode.TestNameNodeOptionParsing   hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.TestFileLengthOnClusterRestart   hadoop.hdfs.server.datanode.TestBPOfferService   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.hdfs.server.datanode.fsdataset.TestAvailableSpaceVolumeChoosingPolicy   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.server.namenode.ha.TestDNFencing JDK v1.7.0_151 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestHSync   org.apache.hadoop.hdfs.TestModTime   org.apache.hadoop.hdfs.TestEncryptionZones   org.apache.hadoop.hdfs.TestLeaseRecovery2   org.apache.hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   org.apache.hadoop.hdfs.server.namenode.TestAllowFormat   org.apache.hadoop.hdfs.TestParallelShortCircuitRead   org.apache.hadoop.hdfs.TestDFSStartupVersions   org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool   org.apache.hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage   org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling   org.apache.hadoop.hdfs.TestFileCreationEmpty   org.apache.hadoop.hdfs.TestFileCreationClient   org.apache.hadoop.hdfs.qjournal.client.TestQuorumJournalManager   org.apache.hadoop.hdfs.TestListFilesInFileContext   org.apache.hadoop.hdfs.TestDatanodeRegistration   org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServer   org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics   org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   org.apache.hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockManager   org.apache.hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade   org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl   org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy   org.apache.hadoop.hdfs.TestBlocksScheduledCounter   org.apache.hadoop.hdfs.server.datanode.TestDataNodeExit   org.apache.hadoop.hdfs.TestDFSClientFailover   org.apache.hadoop.hdfs.TestMultiThreadedHflush   org.apache.hadoop.hdfs.TestSetrepDecreasing   org.apache.hadoop.hdfs.TestReservedRawPaths   org.apache.hadoop.hdfs.TestSetrepIncreasing   org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery   org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement   org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   org.apache.hadoop.hdfs.server.datanode.TestReadOnlySharedStorage   org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes   org.apache.hadoop.hdfs.TestFileCreation   org.apache.hadoop.hdfs.TestQuota   org.apache.hadoop.hdfs.qjournal.server.TestJournalNode   org.apache.hadoop.hdfs.server.namenode.TestINodeFile   org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl   org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery   org.apache.hadoop.hdfs.TestReplication   org.apache.hadoop.hdfs.TestDataTransferKeepalive   org.apache.hadoop.hdfs.TestDatanodeDeath   org.apache.hadoop.hdfs.server.datanode.TestTriggerBlockReport   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   org.apache.hadoop.hdfs.TestPread   org.apache.hadoop.hdfs.TestFileAppend   org.apache.hadoop.hdfs.TestSafeMode   org.apache.hadoop.hdfs.TestDFSClientRetries   org.apache.hadoop.hdfs.TestBlockMissingException   org.apache.hadoop.hdfs.TestDFSFinalize   org.apache.hadoop.hdfs.TestDFSStorageStateRecovery   org.apache.hadoop.hdfs.TestFileAppend4   org.apache.hadoop.hdfs.server.namenode.TestStreamFile   org.apache.hadoop.hdfs.TestFileAppend3   org.apache.hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys   org.apache.hadoop.hdfs.TestFileAppend2   org.apache.hadoop.hdfs.TestRead   org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults   org.apache.hadoop.hdfs.TestDFSUpgradeFromImage   org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade   org.apache.hadoop.hdfs.TestHDFSFileSystemContract   org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage   org.apache.hadoop.hdfs.TestDFSPermission   org.apache.hadoop.hdfs.web.TestWebHdfsTokens   org.apache.hadoop.hdfs.server.datanode.TestBatchIbr   org.apache.hadoop.hdfs.TestDecommission   org.apache.hadoop.hdfs.TestReadWhileWriting   org.apache.hadoop.hdfs.security.TestDelegationToken   org.apache.hadoop.hdfs.TestFileAppendRestart   org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   org.apache.hadoop.hdfs.TestFileCorruption   org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks   org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks   org.apache.hadoop.hdfs.TestCrcCorruption   org.apache.hadoop.hdfs.TestFileCreationDelete   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol   org.apache.hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer   org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS   org.apache.hadoop.hdfs.TestDFSAddressConfig   org.apache.hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM   org.apache.hadoop.hdfs.TestLeaseRecovery   org.apache.hadoop.hdfs.TestBlockStoragePolicy   org.apache.hadoop.hdfs.TestRollingUpgrade   org.apache.hadoop.hdfs.TestDFSRemove   org.apache.hadoop.hdfs.TestSeekBug   org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality   org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks   org.apache.hadoop.hdfs.TestLocalDFS   org.apache.hadoop.hdfs.TestDFSMkdirs   org.apache.hadoop.hdfs.TestHDFSServerPorts   org.apache.hadoop.hdfs.TestLease   org.apache.hadoop.hdfs.TestDFSUpgrade   org.apache.hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes   org.apache.hadoop.hdfs.server.namenode.TestMetaSave   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles   org.apache.hadoop.hdfs.qjournal.TestNNWithQJM   org.apache.hadoop.hdfs.web.TestWebHDFS   org.apache.hadoop.hdfs.TestAppendSnapshotTruncate   org.apache.hadoop.hdfs.tools.TestGetGroups   org.apache.hadoop.hdfs.tools.TestDFSHAAdminMiniCluster   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestRbwSpaceReservation   org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount   org.apache.hadoop.hdfs.TestBlockReaderLocal   org.apache.hadoop.hdfs.web.TestWebHDFSXAttr   org.apache.hadoop.hdfs.TestRollingUpgradeRollback   org.apache.hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController   org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   org.apache.hadoop.hdfs.server.datanode.TestDiskError   org.apache.hadoop.hdfs.TestRestartDFS   org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes   org.apache.hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold   org.apache.hadoop.hdfs.server.datanode.TestIncrementalBlockReports   org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes   org.apache.hadoop.hdfs.server.datanode.TestIncrementalBrVariations   org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache   org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup   org.apache.hadoop.hdfs.TestRenameWhileOpen   org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache   org.apache.hadoop.hdfs.server.datanode.TestTransferRbw   org.apache.hadoop.hdfs.TestFileConcurrentReader   org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   org.apache.hadoop.hdfs.server.namenode.TestXAttrConfigFlag   org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner   org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication   org.apache.hadoop.hdfs.TestFSOutputSummer   org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead   org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover   org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation   org.apache.hadoop.hdfs.TestDFSRollback   org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   org.apache.hadoop.hdfs.TestMiniDFSCluster   org.apache.hadoop.hdfs.server.datanode.TestBlockScanner   org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs   org.apache.hadoop.hdfs.TestBlockReaderFactory   org.apache.hadoop.hdfs.qjournal.client.TestEpochsAreUnique   org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy   org.apache.hadoop.hdfs.TestDistributedFileSystem   org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   org.apache.hadoop.hdfs.server.blockmanagement.TestSequentialBlockId   org.apache.hadoop.hdfs.TestAppendDifferentChecksum   org.apache.hadoop.hdfs.TestHFlush   org.apache.hadoop.hdfs.web.TestWebHDFSForHA   org.apache.hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   org.apache.hadoop.hdfs.TestBalancerBandwidth   org.apache.hadoop.hdfs.TestClientReportBadBlock   org.apache.hadoop.hdfs.TestDFSClientExcludedNodes   org.apache.hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages   org.apache.hadoop.hdfs.TestSetTimes   org.apache.hadoop.hdfs.TestDFSShell   org.apache.hadoop.hdfs.server.namenode.TestHostsFiles   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS   org.apache.hadoop.hdfs.server.datanode.TestCachingStrategy   org.apache.hadoop.hdfs.tools.TestDebugAdmin   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart   org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode   org.apache.hadoop.hdfs.TestDFSRename   org.apache.hadoop.hdfs.tools.TestDFSAdmin   org.apache.hadoop.hdfs.TestGetBlocks   org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   org.apache.hadoop.hdfs.TestGetFileChecksum   org.apache.hadoop.hdfs.TestAbandonBlock   org.apache.hadoop.hdfs.TestDFSShellGenericOptions   org.apache.hadoop.hdfs.web.TestWebHDFSAcl Subsystem Report/Notes Docker Image:yetus/hadoop:67e87c9 JIRA Issue HDFS-8865 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12889763/HDFS-8865.branch-2.7.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml uname Linux e3f1f9830046 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 90aae39 Default Java 1.7.0_151 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_144 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_151 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21443/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/21443/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/21443/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_151.txt JDK v1.7.0_151 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21443/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/21443/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21443/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
                branch-2.7 Compile Tests
          +1 mvninstall 7m 57s branch-2.7 passed
          +1 compile 1m 5s branch-2.7 passed with JDK v1.8.0_144
          +1 compile 1m 4s branch-2.7 passed with JDK v1.7.0_151
          +1 checkstyle 0m 29s branch-2.7 passed
          +1 mvnsite 1m 1s branch-2.7 passed
          +1 findbugs 2m 55s branch-2.7 passed
          +1 javadoc 1m 1s branch-2.7 passed with JDK v1.8.0_144
          +1 javadoc 1m 44s branch-2.7 passed with JDK v1.7.0_151
                Patch Compile Tests
          +1 mvninstall 0m 50s the patch passed
          +1 compile 0m 59s the patch passed with JDK v1.8.0_144
          +1 javac 0m 59s the patch passed
          +1 compile 0m 58s the patch passed with JDK v1.7.0_151
          +1 javac 0m 58s the patch passed
          -0 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 611 unchanged - 3 fixed = 612 total (was 614)
          +1 mvnsite 0m 54s the patch passed
          -1 whitespace 0m 0s The patch has 60 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 3m 3s the patch passed
          +1 javadoc 0m 57s the patch passed with JDK v1.8.0_144
          +1 javadoc 1m 46s the patch passed with JDK v1.7.0_151
                Other Tests
          -1 unit 537m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_151.
          -1 asflicense 6m 24s The patch generated 1 ASF License warnings.
          623m 54s



          Reason Tests
          JDK v1.8.0_144 Failed junit tests hadoop.hdfs.server.namenode.TestDeadDatanode
            hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.web.TestHttpsFileSystem
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          JDK v1.7.0_151 Failed junit tests hadoop.hdfs.TestRead
            hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
            hadoop.hdfs.server.namenode.snapshot.TestSnapshot
            hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController
            hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
            hadoop.hdfs.TestHdfsAdmin
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
            hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler
            hadoop.hdfs.server.namenode.TestCommitBlockWithInvalidGenStamp
            hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            hadoop.hdfs.server.namenode.TestFSEditLogLoader
            hadoop.hdfs.server.namenode.TestAuditLogger
            hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
            hadoop.hdfs.server.datanode.TestDataNodeMetrics
            hadoop.hdfs.TestAppendSnapshotTruncate
            hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
            hadoop.hdfs.server.namenode.TestNameNodeRpcServer
            hadoop.hdfs.TestFileAppendRestart
            hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
            hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
            hadoop.hdfs.server.namenode.ha.TestHAFsck
            hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.server.namenode.TestHDFSConcat
            hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
            hadoop.TestRefreshCallQueue
            hadoop.hdfs.server.namenode.TestAddBlock
            hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold
            hadoop.security.TestPermissionSymlinks
            hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
            hadoop.hdfs.server.namenode.ha.TestHASafeMode
            hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica
            hadoop.hdfs.server.datanode.TestDataNodeExit
            hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA
            hadoop.security.TestRefreshUserMappings
            hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys
            hadoop.hdfs.server.balancer.TestBalancerRPCDelay
            hadoop.hdfs.server.namenode.TestMetadataVersionOutput
            hadoop.hdfs.server.namenode.ha.TestHAMetrics
            hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes
            hadoop.hdfs.server.namenode.ha.TestDNFencing
            hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA
            hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.server.namenode.TestDeadDatanode
            hadoop.hdfs.server.namenode.TestStreamFile
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
            hadoop.hdfs.server.namenode.TestAuditLogs
            hadoop.hdfs.crypto.TestHdfsCryptoStreams
            hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot
            hadoop.hdfs.server.namenode.TestFSDirectory
            hadoop.hdfs.server.datanode.TestTriggerBlockReport
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
            hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
            hadoop.hdfs.server.datanode.TestIncrementalBlockReports
            hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
            hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.server.datanode.TestReadOnlySharedStorage
            hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
            hadoop.hdfs.server.namenode.TestEditLogJournalFailures
            hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
            hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages
            hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
            hadoop.hdfs.server.datanode.TestIncrementalBrVariations
            hadoop.hdfs.TestLeaseRecovery2
            hadoop.hdfs.server.namenode.TestAclConfigFlag
            hadoop.cli.TestXAttrCLI
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename
            hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
            hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure
            hadoop.hdfs.TestListPathServlet
            hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean
            hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
            hadoop.hdfs.server.namenode.TestFSImageWithXAttr
            hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer
            hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA
            hadoop.hdfs.server.datanode.TestDiskError
            hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.security.TestPermission
            hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
            hadoop.hdfs.server.datanode.TestDataNodeInitStorage
            hadoop.hdfs.server.namenode.TestINodeFile
            hadoop.hdfs.server.namenode.ha.TestNNHealthCheck
            hadoop.hdfs.TestIsMethodSupported
            hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.TestFileCreationDelete
            hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
            hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport
            hadoop.hdfs.server.namenode.TestXAttrConfigFlag
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
            hadoop.hdfs.server.blockmanagement.TestNodeCount
            hadoop.hdfs.util.TestDiff
            hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover
            hadoop.hdfs.TestAppendDifferentChecksum
            hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
            hadoop.hdfs.server.datanode.fsdataset.TestAvailableSpaceVolumeChoosingPolicy
            hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
            hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage
            hadoop.hdfs.tools.TestDFSAdminWithHA
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
            hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation
            hadoop.cli.TestAclCLI
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.cli.TestCryptoAdminCLI
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
            hadoop.hdfs.server.namenode.TestTruncateQuotaUpdate
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
            hadoop.hdfs.server.datanode.TestStorageReport
            hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
            hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.datanode.TestDeleteBlockPool
            hadoop.hdfs.TestBlockReaderLocalLegacy
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.TestEditLogAutoroll
            hadoop.hdfs.security.token.block.TestBlockToken
            hadoop.hdfs.server.namenode.TestNameNodeXAttr
            hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
            hadoop.hdfs.util.TestByteArrayManager
            hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
            hadoop.cli.TestCacheAdminCLI
            hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
            hadoop.hdfs.server.namenode.TestFSNamesystemMBean
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing
            hadoop.hdfs.server.blockmanagement.TestSequentialBlockId
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.hdfs.TestDatanodeDeath
            hadoop.hdfs.server.datanode.TestFsDatasetCache
            hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.datanode.fsdataset.impl.TestRbwSpaceReservation
            hadoop.hdfs.security.TestDelegationToken
            hadoop.hdfs.server.datanode.TestCachingStrategy
            hadoop.hdfs.server.namenode.TestFsck
            hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
            hadoop.hdfs.TestSnapshotCommands
            hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
            hadoop.hdfs.server.datanode.TestHSync
            hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA
            hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
            hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.TestDFSOutputStream
            hadoop.hdfs.server.namenode.TestNameNodeAcl
            hadoop.hdfs.server.namenode.TestQuotaByStorageType
            hadoop.hdfs.server.datanode.TestDataNodeMXBean
            hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
            hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.TestDFSInotifyEventInputStream
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication
            hadoop.hdfs.server.namenode.TestNameNodeMXBean
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir
            hadoop.hdfs.server.datanode.TestTransferRbw
            hadoop.hdfs.server.namenode.TestFileContextXAttr
            hadoop.hdfs.server.namenode.TestHostsFiles
            hadoop.hdfs.TestFileCorruption
            hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
            hadoop.hdfs.server.namenode.TestINodeAttributeProvider
          JDK v1.7.0_151 Timed out junit tests org.apache.hadoop.hdfs.TestEncryptionZones
            org.apache.hadoop.hdfs.TestModTime
            org.apache.hadoop.hdfs.server.namenode.TestStartup
            org.apache.hadoop.hdfs.TestDFSStartupVersions
            org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
            org.apache.hadoop.hdfs.TestWriteRead
            org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
            org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
            org.apache.hadoop.hdfs.TestDatanodeRegistration
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServer
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
            org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl
            org.apache.hadoop.hdfs.TestMultiThreadedHflush
            org.apache.hadoop.hdfs.TestDFSClientFailover
            org.apache.hadoop.fs.TestEnhancedByteBufferAccess
            org.apache.hadoop.hdfs.TestReservedRawPaths
            org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage
            org.apache.hadoop.hdfs.TestSetrepIncreasing
            org.apache.hadoop.hdfs.TestSetrepDecreasing
            org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
            org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery
            org.apache.hadoop.hdfs.server.namenode.TestAddBlockRetry
            org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
            org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes
            org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
            org.apache.hadoop.hdfs.server.namenode.TestLeaseManager
            org.apache.hadoop.hdfs.qjournal.server.TestJournalNode
            org.apache.hadoop.hdfs.TestQuota
            org.apache.hadoop.hdfs.TestFileCreation
            org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery
            org.apache.hadoop.hdfs.server.namenode.TestEditLog
            org.apache.hadoop.hdfs.TestReplication
            org.apache.hadoop.hdfs.TestDataTransferKeepalive
            org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
            org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
            org.apache.hadoop.hdfs.TestFileAppend
            org.apache.hadoop.hdfs.TestPread
            org.apache.hadoop.hdfs.TestListFilesInDFS
            org.apache.hadoop.hdfs.server.namenode.TestBackupNode
            org.apache.hadoop.hdfs.server.namenode.TestFSImage
            org.apache.hadoop.hdfs.TestSafeMode
            org.apache.hadoop.hdfs.TestDFSClientRetries
            org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
            org.apache.hadoop.hdfs.TestFileAppend4
            org.apache.hadoop.hdfs.TestDFSFinalize
            org.apache.hadoop.hdfs.TestFileAppend3
            org.apache.hadoop.hdfs.TestFileAppend2
            org.apache.hadoop.hdfs.TestHDFSFileSystemContract
            org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
            org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
            org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults
            org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade
            org.apache.hadoop.hdfs.TestDFSPermission
            org.apache.hadoop.hdfs.web.TestWebHdfsTokens
            org.apache.hadoop.hdfs.server.datanode.TestBatchIbr
            org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives
            org.apache.hadoop.hdfs.TestDecommission
            org.apache.hadoop.hdfs.TestReadWhileWriting
            org.apache.hadoop.hdfs.server.namenode.TestFileLimit
            org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
            org.apache.hadoop.hdfs.TestDFSAddressConfig
            org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
            org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM
            org.apache.hadoop.hdfs.TestRollingUpgrade
            org.apache.hadoop.hdfs.TestSeekBug
            org.apache.hadoop.hdfs.TestDFSRemove
            org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
            org.apache.hadoop.hdfs.server.mover.TestStorageMover
            org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
            org.apache.hadoop.hdfs.TestDFSMkdirs
            org.apache.hadoop.hdfs.TestLocalDFS
            org.apache.hadoop.hdfs.TestHDFSServerPorts
            org.apache.hadoop.hdfs.TestDFSUpgrade
            org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
            org.apache.hadoop.hdfs.TestLease
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
            org.apache.hadoop.hdfs.server.namenode.TestMetaSave
            org.apache.hadoop.hdfs.TestDatanodeReport
            org.apache.hadoop.hdfs.qjournal.TestNNWithQJM
            org.apache.hadoop.hdfs.web.TestWebHDFS
            org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
            org.apache.hadoop.hdfs.tools.TestGetGroups
            org.apache.hadoop.hdfs.tools.TestDFSHAAdminMiniCluster
            org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
            org.apache.hadoop.hdfs.web.TestWebHDFSXAttr
            org.apache.hadoop.hdfs.TestBlockReaderLocal
            org.apache.hadoop.hdfs.TestRollingUpgradeRollback
            org.apache.hadoop.hdfs.TestRestartDFS
            org.apache.hadoop.hdfs.server.mover.TestMover
            org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes
            org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
            org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup
            org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache
            org.apache.hadoop.hdfs.TestPipelines
            org.apache.hadoop.hdfs.TestRenameWhileOpen
            org.apache.hadoop.hdfs.TestFileConcurrentReader
            org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication
            org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl
            org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
            org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
            org.apache.hadoop.hdfs.TestFSOutputSummer
            org.apache.hadoop.hdfs.TestDFSRollback
            org.apache.hadoop.hdfs.server.namenode.TestDeleteRace
            org.apache.hadoop.hdfs.TestMiniDFSCluster
            org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
            org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
            org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
            org.apache.hadoop.hdfs.TestDistributedFileSystem
            org.apache.hadoop.hdfs.TestBlockReaderFactory
            org.apache.hadoop.hdfs.TestHFlush
            org.apache.hadoop.hdfs.web.TestWebHDFSForHA
            org.apache.hadoop.hdfs.TestDFSClientExcludedNodes
            org.apache.hadoop.hdfs.TestClientReportBadBlock
            org.apache.hadoop.hdfs.TestEncryptedTransfer
            org.apache.hadoop.hdfs.TestBalancerBandwidth
            org.apache.hadoop.hdfs.TestDFSShell
            org.apache.hadoop.hdfs.TestSetTimes
            org.apache.hadoop.hdfs.web.TestHftpFileSystem
            org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
            org.apache.hadoop.hdfs.TestPersistBlocks
            org.apache.hadoop.hdfs.tools.TestDebugAdmin
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
            org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics
            org.apache.hadoop.hdfs.TestGetBlocks
            org.apache.hadoop.hdfs.TestDFSRename
            org.apache.hadoop.cli.TestHDFSCLI
            org.apache.hadoop.hdfs.TestAbandonBlock
            org.apache.hadoop.hdfs.web.TestWebHDFSAcl



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:67e87c9
          JIRA Issue HDFS-8865
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12889763/HDFS-8865.branch-2.7.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
          uname Linux 506ea002449b 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 90aae39
          Default Java 1.7.0_151
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_144 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_151
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21456/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/21456/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/21456/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_151.txt
          JDK v1.7.0_151 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21456/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/21456/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21456/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.       branch-2.7 Compile Tests +1 mvninstall 7m 57s branch-2.7 passed +1 compile 1m 5s branch-2.7 passed with JDK v1.8.0_144 +1 compile 1m 4s branch-2.7 passed with JDK v1.7.0_151 +1 checkstyle 0m 29s branch-2.7 passed +1 mvnsite 1m 1s branch-2.7 passed +1 findbugs 2m 55s branch-2.7 passed +1 javadoc 1m 1s branch-2.7 passed with JDK v1.8.0_144 +1 javadoc 1m 44s branch-2.7 passed with JDK v1.7.0_151       Patch Compile Tests +1 mvninstall 0m 50s the patch passed +1 compile 0m 59s the patch passed with JDK v1.8.0_144 +1 javac 0m 59s the patch passed +1 compile 0m 58s the patch passed with JDK v1.7.0_151 +1 javac 0m 58s the patch passed -0 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 611 unchanged - 3 fixed = 612 total (was 614) +1 mvnsite 0m 54s the patch passed -1 whitespace 0m 0s The patch has 60 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 3m 3s the patch passed +1 javadoc 0m 57s the patch passed with JDK v1.8.0_144 +1 javadoc 1m 46s the patch passed with JDK v1.7.0_151       Other Tests -1 unit 537m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_151. -1 asflicense 6m 24s The patch generated 1 ASF License warnings. 623m 54s Reason Tests JDK v1.8.0_144 Failed junit tests hadoop.hdfs.server.namenode.TestDeadDatanode   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.web.TestHttpsFileSystem   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots JDK v1.7.0_151 Failed junit tests hadoop.hdfs.TestRead   hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics   hadoop.hdfs.server.namenode.snapshot.TestSnapshot   hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController   hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd   hadoop.hdfs.TestHdfsAdmin   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler   hadoop.hdfs.server.namenode.TestCommitBlockWithInvalidGenStamp   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.server.namenode.TestFSEditLogLoader   hadoop.hdfs.server.namenode.TestAuditLogger   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.TestAppendSnapshotTruncate   hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup   hadoop.hdfs.server.namenode.TestNameNodeRpcServer   hadoop.hdfs.TestFileAppendRestart   hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade   hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot   hadoop.hdfs.server.namenode.ha.TestHAFsck   hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.namenode.TestHDFSConcat   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics   hadoop.TestRefreshCallQueue   hadoop.hdfs.server.namenode.TestAddBlock   hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold   hadoop.security.TestPermissionSymlinks   hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica   hadoop.hdfs.server.datanode.TestDataNodeExit   hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA   hadoop.security.TestRefreshUserMappings   hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys   hadoop.hdfs.server.balancer.TestBalancerRPCDelay   hadoop.hdfs.server.namenode.TestMetadataVersionOutput   hadoop.hdfs.server.namenode.ha.TestHAMetrics   hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes   hadoop.hdfs.server.namenode.ha.TestDNFencing   hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA   hadoop.hdfs.server.namenode.TestFSImageWithSnapshot   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.server.namenode.TestDeadDatanode   hadoop.hdfs.server.namenode.TestStreamFile   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport   hadoop.hdfs.server.namenode.TestAuditLogs   hadoop.hdfs.crypto.TestHdfsCryptoStreams   hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot   hadoop.hdfs.server.namenode.TestFSDirectory   hadoop.hdfs.server.datanode.TestTriggerBlockReport   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks   hadoop.hdfs.server.datanode.TestIncrementalBlockReports   hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.server.datanode.TestReadOnlySharedStorage   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.server.namenode.TestLargeDirectoryDelete   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.server.namenode.TestSecurityTokenEditLog   hadoop.hdfs.server.namenode.TestEditLogJournalFailures   hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.datanode.TestIncrementalBrVariations   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.server.namenode.TestAclConfigFlag   hadoop.cli.TestXAttrCLI   hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename   hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork   hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure   hadoop.hdfs.TestListPathServlet   hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean   hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer   hadoop.hdfs.server.namenode.TestFSImageWithXAttr   hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer   hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA   hadoop.hdfs.server.datanode.TestDiskError   hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.security.TestPermission   hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot   hadoop.hdfs.server.datanode.TestDataNodeInitStorage   hadoop.hdfs.server.namenode.TestINodeFile   hadoop.hdfs.server.namenode.ha.TestNNHealthCheck   hadoop.hdfs.TestIsMethodSupported   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot   hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport   hadoop.hdfs.server.namenode.TestXAttrConfigFlag   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens   hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.util.TestDiff   hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover   hadoop.hdfs.TestAppendDifferentChecksum   hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps   hadoop.hdfs.server.datanode.fsdataset.TestAvailableSpaceVolumeChoosingPolicy   hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes   hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation   hadoop.cli.TestAclCLI   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.cli.TestCryptoAdminCLI   hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap   hadoop.hdfs.server.namenode.TestTruncateQuotaUpdate   hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters   hadoop.hdfs.server.datanode.TestStorageReport   hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   hadoop.hdfs.server.namenode.TestNameNodeResourceChecker   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.datanode.TestDeleteBlockPool   hadoop.hdfs.TestBlockReaderLocalLegacy   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.TestEditLogAutoroll   hadoop.hdfs.security.token.block.TestBlockToken   hadoop.hdfs.server.namenode.TestNameNodeXAttr   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.util.TestByteArrayManager   hadoop.hdfs.server.blockmanagement.TestReplicationPolicy   hadoop.cli.TestCacheAdminCLI   hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot   hadoop.hdfs.server.namenode.TestFSNamesystemMBean   hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing   hadoop.hdfs.server.blockmanagement.TestSequentialBlockId   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.TestDatanodeDeath   hadoop.hdfs.server.datanode.TestFsDatasetCache   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.fsdataset.impl.TestRbwSpaceReservation   hadoop.hdfs.security.TestDelegationToken   hadoop.hdfs.server.datanode.TestCachingStrategy   hadoop.hdfs.server.namenode.TestFsck   hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   hadoop.hdfs.TestSnapshotCommands   hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks   hadoop.hdfs.server.datanode.TestHSync   hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA   hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing   hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.TestDFSOutputStream   hadoop.hdfs.server.namenode.TestNameNodeAcl   hadoop.hdfs.server.namenode.TestQuotaByStorageType   hadoop.hdfs.server.datanode.TestDataNodeMXBean   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.TestDFSInotifyEventInputStream   hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication   hadoop.hdfs.server.namenode.TestNameNodeMXBean   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir   hadoop.hdfs.server.datanode.TestTransferRbw   hadoop.hdfs.server.namenode.TestFileContextXAttr   hadoop.hdfs.server.namenode.TestHostsFiles   hadoop.hdfs.TestFileCorruption   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate   hadoop.hdfs.server.namenode.TestINodeAttributeProvider JDK v1.7.0_151 Timed out junit tests org.apache.hadoop.hdfs.TestEncryptionZones   org.apache.hadoop.hdfs.TestModTime   org.apache.hadoop.hdfs.server.namenode.TestStartup   org.apache.hadoop.hdfs.TestDFSStartupVersions   org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement   org.apache.hadoop.hdfs.TestWriteRead   org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer   org.apache.hadoop.hdfs.TestDatanodeRegistration   org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   org.apache.hadoop.hdfs.server.namenode.TestNameNodeHttpServer   org.apache.hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade   org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl   org.apache.hadoop.hdfs.TestMultiThreadedHflush   org.apache.hadoop.hdfs.TestDFSClientFailover   org.apache.hadoop.fs.TestEnhancedByteBufferAccess   org.apache.hadoop.hdfs.TestReservedRawPaths   org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage   org.apache.hadoop.hdfs.TestSetrepIncreasing   org.apache.hadoop.hdfs.TestSetrepDecreasing   org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery   org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery   org.apache.hadoop.hdfs.server.namenode.TestAddBlockRetry   org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure   org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes   org.apache.hadoop.hdfs.server.namenode.TestStorageRestore   org.apache.hadoop.hdfs.server.namenode.TestLeaseManager   org.apache.hadoop.hdfs.qjournal.server.TestJournalNode   org.apache.hadoop.hdfs.TestQuota   org.apache.hadoop.hdfs.TestFileCreation   org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery   org.apache.hadoop.hdfs.server.namenode.TestEditLog   org.apache.hadoop.hdfs.TestReplication   org.apache.hadoop.hdfs.TestDataTransferKeepalive   org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace   org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart   org.apache.hadoop.hdfs.TestFileAppend   org.apache.hadoop.hdfs.TestPread   org.apache.hadoop.hdfs.TestListFilesInDFS   org.apache.hadoop.hdfs.server.namenode.TestBackupNode   org.apache.hadoop.hdfs.server.namenode.TestFSImage   org.apache.hadoop.hdfs.TestSafeMode   org.apache.hadoop.hdfs.TestDFSClientRetries   org.apache.hadoop.hdfs.TestDFSStorageStateRecovery   org.apache.hadoop.hdfs.TestFileAppend4   org.apache.hadoop.hdfs.TestDFSFinalize   org.apache.hadoop.hdfs.TestFileAppend3   org.apache.hadoop.hdfs.TestFileAppend2   org.apache.hadoop.hdfs.TestHDFSFileSystemContract   org.apache.hadoop.hdfs.TestDFSUpgradeFromImage   org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage   org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults   org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade   org.apache.hadoop.hdfs.TestDFSPermission   org.apache.hadoop.hdfs.web.TestWebHdfsTokens   org.apache.hadoop.hdfs.server.datanode.TestBatchIbr   org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives   org.apache.hadoop.hdfs.TestDecommission   org.apache.hadoop.hdfs.TestReadWhileWriting   org.apache.hadoop.hdfs.server.namenode.TestFileLimit   org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol   org.apache.hadoop.hdfs.TestDFSAddressConfig   org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS   org.apache.hadoop.hdfs.qjournal.TestSecureNNWithQJM   org.apache.hadoop.hdfs.TestRollingUpgrade   org.apache.hadoop.hdfs.TestSeekBug   org.apache.hadoop.hdfs.TestDFSRemove   org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks   org.apache.hadoop.hdfs.server.mover.TestStorageMover   org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks   org.apache.hadoop.hdfs.TestDFSMkdirs   org.apache.hadoop.hdfs.TestLocalDFS   org.apache.hadoop.hdfs.TestHDFSServerPorts   org.apache.hadoop.hdfs.TestDFSUpgrade   org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite   org.apache.hadoop.hdfs.TestLease   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles   org.apache.hadoop.hdfs.server.namenode.TestMetaSave   org.apache.hadoop.hdfs.TestDatanodeReport   org.apache.hadoop.hdfs.qjournal.TestNNWithQJM   org.apache.hadoop.hdfs.web.TestWebHDFS   org.apache.hadoop.hdfs.server.namenode.TestCheckpoint   org.apache.hadoop.hdfs.tools.TestGetGroups   org.apache.hadoop.hdfs.tools.TestDFSHAAdminMiniCluster   org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs   org.apache.hadoop.hdfs.web.TestWebHDFSXAttr   org.apache.hadoop.hdfs.TestBlockReaderLocal   org.apache.hadoop.hdfs.TestRollingUpgradeRollback   org.apache.hadoop.hdfs.TestRestartDFS   org.apache.hadoop.hdfs.server.mover.TestMover   org.apache.hadoop.hdfs.server.namenode.TestSnapshotPathINodes   org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes   org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup   org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache   org.apache.hadoop.hdfs.TestPipelines   org.apache.hadoop.hdfs.TestRenameWhileOpen   org.apache.hadoop.hdfs.TestFileConcurrentReader   org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication   org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl   org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead   org.apache.hadoop.hdfs.TestFSOutputSummer   org.apache.hadoop.hdfs.TestDFSRollback   org.apache.hadoop.hdfs.server.namenode.TestDeleteRace   org.apache.hadoop.hdfs.TestMiniDFSCluster   org.apache.hadoop.hdfs.server.namenode.TestEditLogRace   org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs   org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction   org.apache.hadoop.hdfs.TestDistributedFileSystem   org.apache.hadoop.hdfs.TestBlockReaderFactory   org.apache.hadoop.hdfs.TestHFlush   org.apache.hadoop.hdfs.web.TestWebHDFSForHA   org.apache.hadoop.hdfs.TestDFSClientExcludedNodes   org.apache.hadoop.hdfs.TestClientReportBadBlock   org.apache.hadoop.hdfs.TestEncryptedTransfer   org.apache.hadoop.hdfs.TestBalancerBandwidth   org.apache.hadoop.hdfs.TestDFSShell   org.apache.hadoop.hdfs.TestSetTimes   org.apache.hadoop.hdfs.web.TestHftpFileSystem   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS   org.apache.hadoop.hdfs.TestPersistBlocks   org.apache.hadoop.hdfs.tools.TestDebugAdmin   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart   org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics   org.apache.hadoop.hdfs.TestGetBlocks   org.apache.hadoop.hdfs.TestDFSRename   org.apache.hadoop.cli.TestHDFSCLI   org.apache.hadoop.hdfs.TestAbandonBlock   org.apache.hadoop.hdfs.web.TestWebHDFSAcl Subsystem Report/Notes Docker Image:yetus/hadoop:67e87c9 JIRA Issue HDFS-8865 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12889763/HDFS-8865.branch-2.7.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml uname Linux 506ea002449b 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 90aae39 Default Java 1.7.0_151 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_144 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_151 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/21456/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/21456/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/21456/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_151.txt JDK v1.7.0_151 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21456/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/21456/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21456/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          INFRA-15194 is failing precommits... Trying to make 2.7 backport executed first.

          Show
          xiaochen Xiao Chen added a comment - INFRA-15194 is failing precommits... Trying to make 2.7 backport executed first.
          Hide
          xiaochen Xiao Chen added a comment -

          IIRC I tried very briefly, but need other changes to make it compile, so stopped there.

          Turns out I was misremembering that with some internal reviews. branch-2.6 patch attached.

          Show
          xiaochen Xiao Chen added a comment - IIRC I tried very briefly, but need other changes to make it compile, so stopped there. Turns out I was misremembering that with some internal reviews. branch-2.6 patch attached.
          Hide
          xiaochen Xiao Chen added a comment -

          hm, https://builds.apache.org/job/PreCommit-hdfs-Build/21423/ seems to confused the branch-2.7 patch and branch-2.6 patch. I see messages saying both of them getting applied... Removing 2.6 and retriggered, hopefully faster than looking at why yetus failed.

          Xiao Chen would you please also add TestDiskspaceQuotaUpdate into the 2.6 patch?

          IIRC I tried very briefly, but need other changes to make it compile, so stopped there.

          Show
          xiaochen Xiao Chen added a comment - hm, https://builds.apache.org/job/PreCommit-hdfs-Build/21423/ seems to confused the branch-2.7 patch and branch-2.6 patch. I see messages saying both of them getting applied... Removing 2.6 and retriggered, hopefully faster than looking at why yetus failed. Xiao Chen would you please also add TestDiskspaceQuotaUpdate into the 2.6 patch? IIRC I tried very briefly, but need other changes to make it compile, so stopped there.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Xiao Chen would you please also add TestDiskspaceQuotaUpdate into the 2.6 patch?

          Show
          jojochuang Wei-Chiu Chuang added a comment - Xiao Chen would you please also add TestDiskspaceQuotaUpdate into the 2.6 patch?
          Hide
          xiaochen Xiao Chen added a comment -

          The 2.6 patch provided by Nicolas Fraison still applies. Re-attaching with minor change in patch name to trigger 2.6 pre-commits.

          Show
          xiaochen Xiao Chen added a comment - The 2.6 patch provided by Nicolas Fraison still applies. Re-attaching with minor change in patch name to trigger 2.6 pre-commits.
          Hide
          xiaochen Xiao Chen added a comment -

          Attached 2.7 patch, which is backported from the branch-2.8 commit.
          Conflicts:
          hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java

          Show
          xiaochen Xiao Chen added a comment - Attached 2.7 patch, which is backported from the branch-2.8 commit. Conflicts: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          Hide
          xiaochen Xiao Chen added a comment -

          Sorry to reopen. Trying to backport this to branch-2.7 and branch-2.6, attaching patches...

          Show
          xiaochen Xiao Chen added a comment - Sorry to reopen. Trying to backport this to branch-2.7 and branch-2.6, attaching patches...
          Hide
          nfraison.criteo Nicolas Fraison added a comment -

          Thanks Kihwal Lee
          Patch for 2.6 branch added

          Show
          nfraison.criteo Nicolas Fraison added a comment - Thanks Kihwal Lee Patch for 2.6 branch added
          Hide
          kihwal Kihwal Lee added a comment -

          Just added you as a HDFS contributor.

          Show
          kihwal Kihwal Lee added a comment - Just added you as a HDFS contributor.
          Hide
          nfraison.criteo Nicolas Fraison added a comment -

          Kihwal Lee I would appreciate to be added as a contributor.

          Even if it is not pushed to any release the backport will be available here for anyone needing it.

          Show
          nfraison.criteo Nicolas Fraison added a comment - Kihwal Lee I would appreciate to be added as a contributor. Even if it is not pushed to any release the backport will be available here for anyone needing it.
          Hide
          kihwal Kihwal Lee added a comment -

          Nicolas Fraison, if you plan to contribute code to the HDFS project in the future, I can add you as a contributor. That will allow you to submit patches. If you want to be a contributor, just let me know.

          As for porting this specific jira to branch-2.6, the release branch for 2.6.5 was just cut, so it will not make the next release. It is also unclear whether 2.6.6 will ever be released.

          Show
          kihwal Kihwal Lee added a comment - Nicolas Fraison , if you plan to contribute code to the HDFS project in the future, I can add you as a contributor. That will allow you to submit patches. If you want to be a contributor, just let me know. As for porting this specific jira to branch-2.6, the release branch for 2.6.5 was just cut, so it will not make the next release. It is also unclear whether 2.6.6 will ever be released.
          Hide
          nfraison.criteo Nicolas Fraison added a comment -

          We have backported this patch to our 2.6 builds.
          Could you please let me knwo how I can provide it, I don't have any rights to attach the git diff file.

          Show
          nfraison.criteo Nicolas Fraison added a comment - We have backported this patch to our 2.6 builds. Could you please let me knwo how I can provide it, I don't have any rights to attach the git diff file.
          Hide
          kihwal Kihwal Lee added a comment -

          We are going to apply this to our 2.7 builds. Here is the patch, in case any one is interested.

          Show
          kihwal Kihwal Lee added a comment - We are going to apply this to our 2.7 builds. Here is the patch, in case any one is interested.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #309 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/309/)
          HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #309 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/309/ ) HDFS-8865 . Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2247 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2247/)
          HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2247 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2247/ ) HDFS-8865 . Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2266 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2266/)
          HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2266 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2266/ ) HDFS-8865 . Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #322 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/322/)
          HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #322 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/322/ ) HDFS-8865 . Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #317 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/317/)
          HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #317 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/317/ ) HDFS-8865 . Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1050 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1050/)
          HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1050 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1050/ ) HDFS-8865 . Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8364 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8364/)
          HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8364 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8364/ ) HDFS-8865 . Improve quota initialization performance. Contributed by Kihwal Lee. (kihwal: rev b6ceee9bf42eec15891f60a014bbfa47e03f563c) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaCounts.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDiskspaceQuotaUpdate.java
          Hide
          kihwal Kihwal Lee added a comment -

          Thanks Xiaoyu Yao and Daryn Sharp for reviews. I've committed this to trunk and branch-2.

          Show
          kihwal Kihwal Lee added a comment - Thanks Xiaoyu Yao and Daryn Sharp for reviews. I've committed this to trunk and branch-2.
          Hide
          daryn Daryn Sharp added a comment -

          +1 This has made a huge difference, and all the possible style warning were addressed.

          Show
          daryn Daryn Sharp added a comment - +1 This has made a huge difference, and all the possible style warning were addressed.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 48s Pre-patch trunk has 4 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 58s There were no new javac warning messages.
          +1 javadoc 9m 52s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 23s The applied patch generated 1 new checkstyle issues (total was 471, now 469).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 27s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 10s Pre-build of native portion
          -1 hdfs tests 188m 5s Tests failed in hadoop-hdfs.
              233m 10s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.TestFSNamesystem
            hadoop.hdfs.server.namenode.TestFileTruncate



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12751597/HDFS-8865.v3.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / a4d9acc
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12140/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12140/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12140/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12140/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12140/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 48s Pre-patch trunk has 4 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 58s There were no new javac warning messages. +1 javadoc 9m 52s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 23s The applied patch generated 1 new checkstyle issues (total was 471, now 469). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 10s Pre-build of native portion -1 hdfs tests 188m 5s Tests failed in hadoop-hdfs.     233m 10s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestFSNamesystem   hadoop.hdfs.server.namenode.TestFileTruncate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12751597/HDFS-8865.v3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a4d9acc Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12140/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12140/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12140/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12140/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12140/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -
          • The checkstyle warning is against DFSConfigKeys.
          • TestEncryptedTransfer failure is not related this patch.
          Show
          kihwal Kihwal Lee added a comment - The checkstyle warning is against DFSConfigKeys. TestEncryptedTransfer failure is not related this patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 50s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 9m 22s There were no new javac warning messages.
          +1 javadoc 11m 41s There were no new javadoc warning messages.
          +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 42s The applied patch generated 1 new checkstyle issues (total was 471, now 469).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 55s mvn install still works.
          +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse.
          +1 findbugs 3m 15s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 42s Pre-build of native portion
          -1 hdfs tests 80m 10s Tests failed in hadoop-hdfs.
              131m 57s  



          Reason Tests
          Timed out tests org.apache.hadoop.hdfs.TestEncryptedTransfer



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12751597/HDFS-8865.v3.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 22de7c1
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12067/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12067/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12067/testReport/
          Java 1.7.0_55
          uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12067/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 50s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 9m 22s There were no new javac warning messages. +1 javadoc 11m 41s There were no new javadoc warning messages. +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 42s The applied patch generated 1 new checkstyle issues (total was 471, now 469). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 55s mvn install still works. +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse. +1 findbugs 3m 15s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 42s Pre-build of native portion -1 hdfs tests 80m 10s Tests failed in hadoop-hdfs.     131m 57s   Reason Tests Timed out tests org.apache.hadoop.hdfs.TestEncryptedTransfer Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12751597/HDFS-8865.v3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 22de7c1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12067/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12067/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12067/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12067/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          Thanks for the review Xiaoyu Yao.
          1. Numbers for small/medium size name space: I do not have numbers for them, but I expect them to be better than before. The test cases show it taking a couple of milliseconds for a very small name space, so the overhead of fork-join doesn't seem to be of too much concern. For very small namespaces, the ones that take far less than a second to initialize, the multithreaded initialization cpuld take a bit longer.

          2. This involves conversion to slf4j. I started down this path, but ended up needing to update IOUtils as well. Since it is not a matter of updating just FSImage, I won't do it in this jira. Instead I will simply wrap it with isDebugEnabled() check for now.

          Show
          kihwal Kihwal Lee added a comment - Thanks for the review Xiaoyu Yao . 1. Numbers for small/medium size name space: I do not have numbers for them, but I expect them to be better than before. The test cases show it taking a couple of milliseconds for a very small name space, so the overhead of fork-join doesn't seem to be of too much concern. For very small namespaces, the ones that take far less than a second to initialize, the multithreaded initialization cpuld take a bit longer. 2. This involves conversion to slf4j. I started down this path, but ended up needing to update IOUtils as well. Since it is not a matter of updating just FSImage , I won't do it in this jira. Instead I will simply wrap it with isDebugEnabled() check for now.
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks for the patch, Kihwal Lee! It looks pretty good to me.

          Just a few comments:
          1. The number for large namespace looks impressive. Do you have the number for small/medium namespace?

          2. Is it possible to add some profiling info between these logs below so that we can easily find how long it takes to finish quota initialization from the log?

          LOG.info("Initializing quota with " + threads + " thread(s)");
          
          ...
          LOG.info("Quota initialization complete.\n" + counts);
          

          3. Can you change to parameterized logging to avoid parameter construction in case the log statement is disabled. For example,

          LOG.debug("Setting quota for {} +\n{}", dir,  myCounts);
          

          4. NIT: typo chached -> cached?

          // Directly access the name system to obtain the current chached usage.
          

          5. Now that HDFS-8879 is in, can you rebase and update the patch? Thanks!

          Show
          xyao Xiaoyu Yao added a comment - Thanks for the patch, Kihwal Lee ! It looks pretty good to me. Just a few comments: 1. The number for large namespace looks impressive. Do you have the number for small/medium namespace? 2. Is it possible to add some profiling info between these logs below so that we can easily find how long it takes to finish quota initialization from the log? LOG.info( "Initializing quota with " + threads + " thread(s)" ); ... LOG.info( "Quota initialization complete.\n" + counts); 3. Can you change to parameterized logging to avoid parameter construction in case the log statement is disabled. For example, LOG.debug( "Setting quota for {} +\n{}" , dir, myCounts); 4. NIT: typo chached -> cached? // Directly access the name system to obtain the current chached usage. 5. Now that HDFS-8879 is in, can you rebase and update the patch? Thanks!
          Hide
          kihwal Kihwal Lee added a comment -

          The checkstyle error is for the new config key, which I am not going to fix.
          The unit test timeout does not happen when I run it. Looks like it is failing in other pre-commit builds too, so it is not being caused by this patch.

          Show
          kihwal Kihwal Lee added a comment - The checkstyle error is for the new config key, which I am not going to fix. The unit test timeout does not happen when I run it. Looks like it is failing in other pre-commit builds too, so it is not being caused by this patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 9s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 41s There were no new javac warning messages.
          +1 javadoc 9m 56s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 21s The applied patch generated 1 new checkstyle issues (total was 493, now 491).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 21s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 2m 38s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 10s Pre-build of native portion
          -1 hdfs tests 186m 4s Tests failed in hadoop-hdfs.
              230m 19s  



          Reason Tests
          Timed out tests org.apache.hadoop.cli.TestHDFSCLI



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12749856/HDFS-8865.v2.checkstyle.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / fa1d84a
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11962/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11962/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11962/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11962/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 9s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 41s There were no new javac warning messages. +1 javadoc 9m 56s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 21s The applied patch generated 1 new checkstyle issues (total was 493, now 491). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 21s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 2m 38s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 10s Pre-build of native portion -1 hdfs tests 186m 4s Tests failed in hadoop-hdfs.     230m 19s   Reason Tests Timed out tests org.apache.hadoop.cli.TestHDFSCLI Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12749856/HDFS-8865.v2.checkstyle.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fa1d84a checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11962/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11962/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11962/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11962/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          Missed the one checkstyle warning.

          Show
          kihwal Kihwal Lee added a comment - Missed the one checkstyle warning.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 40s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 42s There were no new javac warning messages.
          +1 javadoc 9m 39s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 21s The applied patch generated 3 new checkstyle issues (total was 491, now 491).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 20s mvn install still works.
          +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
          +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 4s Pre-build of native portion
          -1 hdfs tests 176m 53s Tests failed in hadoop-hdfs.
              221m 11s  



          Reason Tests
          Timed out tests org.apache.hadoop.cli.TestHDFSCLI



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12749585/HDFS-8865.v2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8f73bdd
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11951/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11951/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11951/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11951/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 40s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 39s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 21s The applied patch generated 3 new checkstyle issues (total was 491, now 491). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 20s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 4s Pre-build of native portion -1 hdfs tests 176m 53s Tests failed in hadoop-hdfs.     221m 11s   Reason Tests Timed out tests org.apache.hadoop.cli.TestHDFSCLI Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12749585/HDFS-8865.v2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8f73bdd checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11951/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11951/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11951/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11951/console This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Kihwal Lee! I agree with your comments above and post a patch on HDFS-8879. This will help us to backport these fixes to 2.7 branches.

          Show
          xyao Xiaoyu Yao added a comment - Thanks Kihwal Lee ! I agree with your comments above and post a patch on HDFS-8879 . This will help us to backport these fixes to 2.7 branches.
          Hide
          kihwal Kihwal Lee added a comment - - edited

          Can we fix the quota by storage type update issue in a separate JIRA?

          It is kind of hard to not do it, since that part of code is being modified. I will just do whatever seems natural in the new code. We can make sure it is fixed properly and improve the test case in HDFS-8879. We can set this jira as a dependency of HDFS-8879, if it ends up containing the fix.

          Show
          kihwal Kihwal Lee added a comment - - edited Can we fix the quota by storage type update issue in a separate JIRA? It is kind of hard to not do it, since that part of code is being modified. I will just do whatever seems natural in the new code. We can make sure it is fixed properly and improve the test case in HDFS-8879 . We can set this jira as a dependency of HDFS-8879 , if it ends up containing the fix.
          Hide
          kihwal Kihwal Lee added a comment - - edited

          New patch

          • Fixed the findbug warning of unused variable.
          • Fixed the test breakages. Snapshot tests were breaking because of casting ref inode to dir. Changed it to use child.asDirectory() when generating work. TestLeaseRecovery2 and TestHDFSCLI are also passing locally.
          • Addressed most checkstyle issues. A couple of them are ignored.
          Show
          kihwal Kihwal Lee added a comment - - edited New patch Fixed the findbug warning of unused variable. Fixed the test breakages. Snapshot tests were breaking because of casting ref inode to dir. Changed it to use child.asDirectory() when generating work. TestLeaseRecovery2 and TestHDFSCLI are also passing locally. Addressed most checkstyle issues. A couple of them are ignored.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 26m 11s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 8m 59s There were no new javac warning messages.
          +1 javadoc 10m 20s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 55s The applied patch generated 10 new checkstyle issues (total was 491, now 498).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
          -1 findbugs 2m 44s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          +1 native 3m 9s Pre-build of native portion
          -1 hdfs tests 175m 32s Tests failed in hadoop-hdfs.
              231m 23s  



          Reason Tests
          FindBugs module:hadoop-hdfs
          Failed unit tests hadoop.hdfs.server.namenode.snapshot.TestSnapshot
            hadoop.hdfs.TestLeaseRecovery2
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          Timed out tests org.apache.hadoop.cli.TestHDFSCLI



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12749347/HDFS-8865.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8f73bdd
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11938/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11938/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11938/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11938/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11938/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 26m 11s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 8m 59s There were no new javac warning messages. +1 javadoc 10m 20s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 55s The applied patch generated 10 new checkstyle issues (total was 491, now 498). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. -1 findbugs 2m 44s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 9s Pre-build of native portion -1 hdfs tests 175m 32s Tests failed in hadoop-hdfs.     231m 23s   Reason Tests FindBugs module:hadoop-hdfs Failed unit tests hadoop.hdfs.server.namenode.snapshot.TestSnapshot   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Timed out tests org.apache.hadoop.cli.TestHDFSCLI Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12749347/HDFS-8865.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8f73bdd checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11938/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11938/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11938/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11938/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11938/console This message was automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          Kihwal Lee, thanks for working on this improvement work and fixing the issue on quota by storage type usage update.
          Can we fix the quota by storage type update issue in a separate JIRA?

          Show
          xyao Xiaoyu Yao added a comment - Kihwal Lee , thanks for working on this improvement work and fixing the issue on quota by storage type usage update. Can we fix the quota by storage type update issue in a separate JIRA?
          Hide
          kihwal Kihwal Lee added a comment -

          While porting the patch from 2.6 to trunk, I noticed something odd in FSImagte.java.

                final EnumCounters<StorageType> typeSpaces =
                    new EnumCounters<StorageType>(StorageType.class);
                for (StorageType t : StorageType.getTypesSupportingQuota()) {
                  final long typeSpace = counts.getTypeSpaces().get(t) -
                      parentTypeSpaces.get(t);
                  final long typeQuota = q.getTypeSpaces().get(t);
                  if (Quota.isViolated(typeQuota, typeSpace)) {
                    LOG.warn("Storage type quota violation in image for "
                        + dir.getFullPathName()
                        + " type = " + t.toString() + " quota = "
                        + typeQuota + " < consumed " + typeSpace);
                  }
                }
          
                dir.getDirectoryWithQuotaFeature().setSpaceConsumed(namespace, ssConsumed,
                    typeSpaces);
          

          typeSpaces is passed to setSpaceConsumed, but it is empty. On restart of NN, storage type-based quota usage tracking will be reset.

          Show
          kihwal Kihwal Lee added a comment - While porting the patch from 2.6 to trunk, I noticed something odd in FSImagte.java . final EnumCounters<StorageType> typeSpaces = new EnumCounters<StorageType>(StorageType.class); for (StorageType t : StorageType.getTypesSupportingQuota()) { final long typeSpace = counts.getTypeSpaces().get(t) - parentTypeSpaces.get(t); final long typeQuota = q.getTypeSpaces().get(t); if (Quota.isViolated(typeQuota, typeSpace)) { LOG.warn( "Storage type quota violation in image for " + dir.getFullPathName() + " type = " + t.toString() + " quota = " + typeQuota + " < consumed " + typeSpace); } } dir.getDirectoryWithQuotaFeature().setSpaceConsumed(namespace, ssConsumed, typeSpaces); typeSpaces is passed to setSpaceConsumed , but it is empty. On restart of NN, storage type-based quota usage tracking will be reset.
          Hide
          kihwal Kihwal Lee added a comment -

          I will post the patch shortly.

          Show
          kihwal Kihwal Lee added a comment - I will post the patch shortly.

            People

            • Assignee:
              kihwal Kihwal Lee
              Reporter:
              kihwal Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development