Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3754

BlockSender doesn't shutdown ReadaheadPool threads

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 2.0.0-alpha
    • Fix Version/s: 1.2.0, 2.0.2-alpha
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The BlockSender doesn't shutdown the ReadaheadPool threads so when tests are run with native libraries some tests fail (time out) because shutdown hangs waiting for the outstanding threads to exit.

      1. hdfs-3754.txt
        3 kB
        Eli Collins
      2. hdfs-3754.txt
        3 kB
        Eli Collins
      3. hdfs-3754.txt
        4 kB
        Eli Collins
      4. hdfs-3754.txt
        4 kB
        Eli Collins
      5. hdfs-3754.txt
        2 kB
        Eli Collins
      6. hdfs-3754.txt
        2 kB
        Eli Collins
      7. hdfs-3754.txt
        2 kB
        Eli Collins
      8. hdfs-3754-b1.txt
        4 kB
        Eli Collins
      9. hdfs-3754-b1.txt
        2 kB
        Eli Collins

        Issue Links

          Activity

          Hide
          Eli Collins added a comment -

          Patch attached. I confirmed TestSeekBug no longer hangs (after using the workaround from HDFS-3753) with this patch.

          The same change applies to branch-1, verified ant clean test -Dtestcase=TestSeekBug -Dcompile.native=true no longer hangs.

          Show
          Eli Collins added a comment - Patch attached. I confirmed TestSeekBug no longer hangs (after using the workaround from HDFS-3753 ) with this patch. The same change applies to branch-1, verified ant clean test -Dtestcase=TestSeekBug -Dcompile.native=true no longer hangs.
          Hide
          Eli Collins added a comment -

          (this test only hangs btw if HDFS-3697 is also applied)

          Show
          Eli Collins added a comment - (this test only hangs btw if HDFS-3697 is also applied)
          Hide
          Suresh Srinivas added a comment -

          Eli, this affect only 1.2.0 release right. Do we have read ahead changes in 1.1.0?

          Show
          Suresh Srinivas added a comment - Eli, this affect only 1.2.0 release right. Do we have read ahead changes in 1.1.0?
          Hide
          Eli Collins added a comment -

          This affects trunk, branch-2 and branch-1.

          Show
          Eli Collins added a comment - This affects trunk, branch-2 and branch-1.
          Hide
          Eli Collins added a comment -

          The only reason mvn -Dtest=TestSeekBug test -Pnative doesn't hang on trunk right now is because HDFS-3753 is preventing the native library from being loaded.

          Show
          Eli Collins added a comment - The only reason mvn -Dtest=TestSeekBug test -Pnative doesn't hang on trunk right now is because HDFS-3753 is preventing the native library from being loaded.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12538927/hdfs-3754.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2938//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2938//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538927/hdfs-3754.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2938//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2938//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          No tests since this unblocks the existing tests.

          Show
          Eli Collins added a comment - No tests since this unblocks the existing tests.
          Hide
          Todd Lipcon added a comment -

          This doesn't look right. The ReadaheadPool is static. To fix this, I think you need to either (a) move the ReadaheadPool inside the DN, and shut it down from the DN context, (b) change the classloading of BlockSender so that the ReadaheadPool doesn't get created inside the Datanode ThreadGroup, or (c) have the DN shutdown method call something like BlockSender.readaheadPool.shutdown()

          Show
          Todd Lipcon added a comment - This doesn't look right. The ReadaheadPool is static. To fix this, I think you need to either (a) move the ReadaheadPool inside the DN, and shut it down from the DN context, (b) change the classloading of BlockSender so that the ReadaheadPool doesn't get created inside the Datanode ThreadGroup, or (c) have the DN shutdown method call something like BlockSender.readaheadPool.shutdown()
          Hide
          Eli Collins added a comment -

          Arg, you're right. Updated patch which takes option c attached.

          Show
          Eli Collins added a comment - Arg, you're right. Updated patch which takes option c attached.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12538946/hdfs-3754.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper
          org.apache.hadoop.hdfs.TestShortCircuitLocalRead
          org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
          org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
          org.apache.hadoop.hdfs.TestFSInputChecker
          org.apache.hadoop.hdfs.server.namenode.TestBackupNode
          org.apache.hadoop.hdfs.TestDataTransferProtocol
          org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat
          org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
          org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
          org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          org.apache.hadoop.hdfs.TestDFSClientFailover
          org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
          org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
          org.apache.hadoop.tools.TestJMXGet
          org.apache.hadoop.hdfs.TestDFSShell
          org.apache.hadoop.hdfs.security.TestDelegationToken
          org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
          org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
          org.apache.hadoop.fs.permission.TestStickyBit
          org.apache.hadoop.hdfs.TestFileConcurrentReader
          org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
          org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
          org.apache.hadoop.fs.TestFcHdfsCreateMkdir
          org.apache.hadoop.hdfs.TestCrcCorruption
          org.apache.hadoop.hdfs.TestAppendDifferentChecksum
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
          org.apache.hadoop.fs.viewfs.TestViewFileSystemHdfs
          org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
          org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser
          org.apache.hadoop.hdfs.TestDFSPermission
          org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
          org.apache.hadoop.hdfs.TestListFilesInFileContext
          org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
          org.apache.hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
          org.apache.hadoop.hdfs.TestDatanodeReport
          org.apache.hadoop.hdfs.TestDFSRemove
          org.apache.hadoop.hdfs.TestRestartDFS
          org.apache.hadoop.hdfs.TestFSOutputSummer
          org.apache.hadoop.hdfs.TestDatanodeRegistration
          org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
          org.apache.hadoop.hdfs.TestHDFSTrash
          org.apache.hadoop.hdfs.TestDFSRollback
          org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
          org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
          org.apache.hadoop.hdfs.TestClientReportBadBlock
          org.apache.hadoop.hdfs.TestDecommission
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeExit
          org.apache.hadoop.hdfs.TestBalancerBandwidth
          org.apache.hadoop.hdfs.TestSmallBlock
          org.apache.hadoop.hdfs.server.datanode.TestHSync
          org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement
          org.apache.hadoop.hdfs.TestFileAppend2
          org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
          org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus
          org.apache.hadoop.hdfs.TestListPathServlet
          org.apache.hadoop.hdfs.server.namenode.TestStreamFile
          org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
          org.apache.hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
          org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
          org.apache.hadoop.hdfs.TestFileCreationDelete
          org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader
          org.apache.hadoop.hdfs.server.namenode.TestAuditLogs
          org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
          org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling
          org.apache.hadoop.hdfs.TestFileAppend
          org.apache.hadoop.hdfs.TestMissingBlocksAlert
          org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing
          org.apache.hadoop.hdfs.TestSeekBug
          org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner
          org.apache.hadoop.hdfs.TestHDFSServerPorts
          org.apache.hadoop.hdfs.TestAbandonBlock
          org.apache.hadoop.hdfs.server.datanode.TestDiskError
          org.apache.hadoop.hdfs.TestRenameWhileOpen
          org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery
          org.apache.hadoop.hdfs.TestBlockReaderLocal
          org.apache.hadoop.cli.TestHDFSCLI
          org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
          org.apache.hadoop.hdfs.TestGetBlocks
          org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
          org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
          org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
          org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
          org.apache.hadoop.fs.viewfs.TestViewFsHdfs
          org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
          org.apache.hadoop.hdfs.TestBlockMissingException
          org.apache.hadoop.hdfs.server.balancer.TestBalancer
          org.apache.hadoop.hdfs.TestQuota
          org.apache.hadoop.fs.TestUrlStreamHandler
          org.apache.hadoop.hdfs.TestFileCreation
          org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode
          org.apache.hadoop.fs.TestHDFSFileContextMainOperations
          org.apache.hadoop.hdfs.TestSetrepDecreasing
          org.apache.hadoop.hdfs.TestFileCreationEmpty
          org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
          org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
          org.apache.hadoop.hdfs.server.datanode.TestBlockReport
          org.apache.hadoop.hdfs.TestDatanodeDeath
          org.apache.hadoop.hdfs.TestParallelRead
          org.apache.hadoop.hdfs.TestFileStatus
          org.apache.hadoop.hdfs.TestFileAppend3
          org.apache.hadoop.hdfs.TestHFlush
          org.apache.hadoop.hdfs.TestIsMethodSupported
          org.apache.hadoop.hdfs.TestListFilesInDFS
          org.apache.hadoop.hdfs.TestPipelines
          org.apache.hadoop.security.TestRefreshUserMappings
          org.apache.hadoop.hdfs.server.datanode.TestDatanodeJsp
          org.apache.hadoop.hdfs.server.datanode.TestTransferRbw
          org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
          org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
          org.apache.hadoop.fs.TestGlobPaths
          org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
          org.apache.hadoop.hdfs.server.namenode.TestMetaSave
          org.apache.hadoop.hdfs.TestDFSStartupVersions
          org.apache.hadoop.hdfs.TestFileCreationClient
          org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes
          org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
          org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica
          org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
          org.apache.hadoop.hdfs.server.journalservice.TestJournalService
          org.apache.hadoop.hdfs.TestLeaseRecovery
          org.apache.hadoop.hdfs.TestLocalDFS
          org.apache.hadoop.security.TestPermission
          org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp
          org.apache.hadoop.fs.TestFcHdfsSymlink
          org.apache.hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA
          org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
          org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
          org.apache.hadoop.hdfs.TestDFSRename
          org.apache.hadoop.hdfs.TestHftpFileSystem
          org.apache.hadoop.hdfs.TestDFSAddressConfig
          org.apache.hadoop.hdfs.TestFetchImage
          org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
          org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
          org.apache.hadoop.hdfs.server.namenode.ha.TestHAMetrics
          org.apache.hadoop.hdfs.TestFileCreationNamenodeRestart
          org.apache.hadoop.hdfs.TestDFSClientExcludedNodes
          org.apache.hadoop.hdfs.TestSetrepIncreasing
          org.apache.hadoop.hdfs.TestLeaseRecovery2
          org.apache.hadoop.hdfs.TestWriteRead
          org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
          org.apache.hadoop.hdfs.TestMiniDFSCluster
          org.apache.hadoop.hdfs.TestFileCorruption
          org.apache.hadoop.hdfs.TestHDFSFileSystemContract
          org.apache.hadoop.hdfs.TestReadWhileWriting
          org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount
          org.apache.hadoop.hdfs.TestDFSFinalize
          org.apache.hadoop.hdfs.TestMultiThreadedHflush
          org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
          org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
          org.apache.hadoop.hdfs.TestDFSUpgrade
          org.apache.hadoop.hdfs.TestDFSMkdirs
          org.apache.hadoop.hdfs.TestDatanodeConfig
          org.apache.hadoop.hdfs.server.namenode.TestStartup
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean
          org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode
          org.apache.hadoop.hdfs.TestReplication
          org.apache.hadoop.hdfs.TestConnCache
          org.apache.hadoop.fs.TestResolveHdfsSymlink
          org.apache.hadoop.hdfs.TestDFSShellGenericOptions
          org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
          org.apache.hadoop.hdfs.TestSafeMode
          org.apache.hadoop.hdfs.web.TestWebHDFS
          org.apache.hadoop.hdfs.TestDFSClientRetries
          org.apache.hadoop.hdfs.TestFileAppend4
          org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend
          org.apache.hadoop.fs.viewfs.TestViewFsAtHdfsRoot
          org.apache.hadoop.hdfs.TestModTime
          org.apache.hadoop.hdfs.TestLease
          org.apache.hadoop.hdfs.TestDataTransferKeepalive
          org.apache.hadoop.hdfs.TestLargeBlock
          org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
          org.apache.hadoop.hdfs.TestClientBlockVerification
          org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
          org.apache.hadoop.hdfs.TestParallelLocalRead
          org.apache.hadoop.hdfs.TestSetTimes
          org.apache.hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
          org.apache.hadoop.hdfs.TestDatanodeBlockScanner
          org.apache.hadoop.hdfs.TestPersistBlocks
          org.apache.hadoop.hdfs.server.namenode.TestFileLimit
          org.apache.hadoop.fs.TestFcHdfsPermission
          org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart
          org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          org.apache.hadoop.hdfs.server.namenode.TestFsck
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          org.apache.hadoop.hdfs.server.namenode.TestAllowFormat
          org.apache.hadoop.hdfs.security.token.block.TestBlockToken
          org.apache.hadoop.hdfs.TestDistributedFileSystem
          org.apache.hadoop.hdfs.TestPread
          org.apache.hadoop.hdfs.TestFileAppendRestart
          org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
          org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2939//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2939//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538946/hdfs-3754.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper org.apache.hadoop.hdfs.TestShortCircuitLocalRead org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs org.apache.hadoop.hdfs.server.namenode.TestStorageRestore org.apache.hadoop.hdfs.TestFSInputChecker org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.TestDataTransferProtocol org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting org.apache.hadoop.hdfs.TestDFSClientFailover org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS org.apache.hadoop.tools.TestJMXGet org.apache.hadoop.hdfs.TestDFSShell org.apache.hadoop.hdfs.security.TestDelegationToken org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication org.apache.hadoop.fs.permission.TestStickyBit org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks org.apache.hadoop.fs.TestFcHdfsCreateMkdir org.apache.hadoop.hdfs.TestCrcCorruption org.apache.hadoop.hdfs.TestAppendDifferentChecksum org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration org.apache.hadoop.fs.viewfs.TestViewFileSystemHdfs org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser org.apache.hadoop.hdfs.TestDFSPermission org.apache.hadoop.hdfs.TestDFSUpgradeFromImage org.apache.hadoop.hdfs.TestListFilesInFileContext org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker org.apache.hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics org.apache.hadoop.hdfs.TestDatanodeReport org.apache.hadoop.hdfs.TestDFSRemove org.apache.hadoop.hdfs.TestRestartDFS org.apache.hadoop.hdfs.TestFSOutputSummer org.apache.hadoop.hdfs.TestDatanodeRegistration org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks org.apache.hadoop.hdfs.TestHDFSTrash org.apache.hadoop.hdfs.TestDFSRollback org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks org.apache.hadoop.hdfs.TestClientReportBadBlock org.apache.hadoop.hdfs.TestDecommission org.apache.hadoop.hdfs.server.datanode.TestDataNodeExit org.apache.hadoop.hdfs.TestBalancerBandwidth org.apache.hadoop.hdfs.TestSmallBlock org.apache.hadoop.hdfs.server.datanode.TestHSync org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus org.apache.hadoop.hdfs.TestListPathServlet org.apache.hadoop.hdfs.server.namenode.TestStreamFile org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart org.apache.hadoop.hdfs.server.namenode.ha.TestQuotasWithHA org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean org.apache.hadoop.hdfs.TestFileCreationDelete org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader org.apache.hadoop.hdfs.server.namenode.TestAuditLogs org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes org.apache.hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling org.apache.hadoop.hdfs.TestFileAppend org.apache.hadoop.hdfs.TestMissingBlocksAlert org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing org.apache.hadoop.hdfs.TestSeekBug org.apache.hadoop.hdfs.server.datanode.TestMultipleNNDataBlockScanner org.apache.hadoop.hdfs.TestHDFSServerPorts org.apache.hadoop.hdfs.TestAbandonBlock org.apache.hadoop.hdfs.server.datanode.TestDiskError org.apache.hadoop.hdfs.TestRenameWhileOpen org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery org.apache.hadoop.hdfs.TestBlockReaderLocal org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner org.apache.hadoop.hdfs.TestGetBlocks org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks org.apache.hadoop.fs.loadGenerator.TestLoadGenerator org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery org.apache.hadoop.fs.viewfs.TestViewFsHdfs org.apache.hadoop.hdfs.server.namenode.TestCheckpoint org.apache.hadoop.hdfs.TestBlockMissingException org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.TestQuota org.apache.hadoop.fs.TestUrlStreamHandler org.apache.hadoop.hdfs.TestFileCreation org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode org.apache.hadoop.fs.TestHDFSFileContextMainOperations org.apache.hadoop.hdfs.TestSetrepDecreasing org.apache.hadoop.hdfs.TestFileCreationEmpty org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled org.apache.hadoop.hdfs.server.datanode.TestBlockReport org.apache.hadoop.hdfs.TestDatanodeDeath org.apache.hadoop.hdfs.TestParallelRead org.apache.hadoop.hdfs.TestFileStatus org.apache.hadoop.hdfs.TestFileAppend3 org.apache.hadoop.hdfs.TestHFlush org.apache.hadoop.hdfs.TestIsMethodSupported org.apache.hadoop.hdfs.TestListFilesInDFS org.apache.hadoop.hdfs.TestPipelines org.apache.hadoop.security.TestRefreshUserMappings org.apache.hadoop.hdfs.server.datanode.TestDatanodeJsp org.apache.hadoop.hdfs.server.datanode.TestTransferRbw org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions org.apache.hadoop.fs.TestGlobPaths org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction org.apache.hadoop.hdfs.server.namenode.TestMetaSave org.apache.hadoop.hdfs.TestDFSStartupVersions org.apache.hadoop.hdfs.TestFileCreationClient org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks org.apache.hadoop.hdfs.server.journalservice.TestJournalService org.apache.hadoop.hdfs.TestLeaseRecovery org.apache.hadoop.hdfs.TestLocalDFS org.apache.hadoop.security.TestPermission org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp org.apache.hadoop.fs.TestFcHdfsSymlink org.apache.hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens org.apache.hadoop.hdfs.TestDFSRename org.apache.hadoop.hdfs.TestHftpFileSystem org.apache.hadoop.hdfs.TestDFSAddressConfig org.apache.hadoop.hdfs.TestFetchImage org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs org.apache.hadoop.hdfs.server.namenode.TestEditLogRace org.apache.hadoop.hdfs.server.namenode.ha.TestHAMetrics org.apache.hadoop.hdfs.TestFileCreationNamenodeRestart org.apache.hadoop.hdfs.TestDFSClientExcludedNodes org.apache.hadoop.hdfs.TestSetrepIncreasing org.apache.hadoop.hdfs.TestLeaseRecovery2 org.apache.hadoop.hdfs.TestWriteRead org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer org.apache.hadoop.hdfs.TestMiniDFSCluster org.apache.hadoop.hdfs.TestFileCorruption org.apache.hadoop.hdfs.TestHDFSFileSystemContract org.apache.hadoop.hdfs.TestReadWhileWriting org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount org.apache.hadoop.hdfs.TestDFSFinalize org.apache.hadoop.hdfs.TestMultiThreadedHflush org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage org.apache.hadoop.hdfs.TestDFSUpgrade org.apache.hadoop.hdfs.TestDFSMkdirs org.apache.hadoop.hdfs.TestDatanodeConfig org.apache.hadoop.hdfs.server.namenode.TestStartup org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode org.apache.hadoop.hdfs.TestReplication org.apache.hadoop.hdfs.TestConnCache org.apache.hadoop.fs.TestResolveHdfsSymlink org.apache.hadoop.hdfs.TestDFSShellGenericOptions org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy org.apache.hadoop.hdfs.TestSafeMode org.apache.hadoop.hdfs.web.TestWebHDFS org.apache.hadoop.hdfs.TestDFSClientRetries org.apache.hadoop.hdfs.TestFileAppend4 org.apache.hadoop.hdfs.server.namenode.ha.TestHAAppend org.apache.hadoop.fs.viewfs.TestViewFsAtHdfsRoot org.apache.hadoop.hdfs.TestModTime org.apache.hadoop.hdfs.TestLease org.apache.hadoop.hdfs.TestDataTransferKeepalive org.apache.hadoop.hdfs.TestLargeBlock org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps org.apache.hadoop.hdfs.TestClientBlockVerification org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure org.apache.hadoop.hdfs.TestParallelLocalRead org.apache.hadoop.hdfs.TestSetTimes org.apache.hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot org.apache.hadoop.hdfs.TestDatanodeBlockScanner org.apache.hadoop.hdfs.TestPersistBlocks org.apache.hadoop.hdfs.server.namenode.TestFileLimit org.apache.hadoop.fs.TestFcHdfsPermission org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover org.apache.hadoop.hdfs.server.namenode.TestFsck org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure org.apache.hadoop.hdfs.server.namenode.TestAllowFormat org.apache.hadoop.hdfs.security.token.block.TestBlockToken org.apache.hadoop.hdfs.TestDistributedFileSystem org.apache.hadoop.hdfs.TestPread org.apache.hadoop.hdfs.TestFileAppendRestart org.apache.hadoop.hdfs.TestWriteConfigurationToDFS org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2939//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2939//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          Right patch this time.

          Show
          Eli Collins added a comment - Right patch this time.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12538979/hdfs-3754.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2941//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2941//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538979/hdfs-3754.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2941//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2941//console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          BlockSender readaheadPool is static. If we shutdown it, could we still able to restart DNs or shutdown some of the DNs when running multiple DNs in unit tests?

          Show
          Tsz Wo Nicholas Sze added a comment - BlockSender readaheadPool is static. If we shutdown it, could we still able to restart DNs or shutdown some of the DNs when running multiple DNs in unit tests?
          Hide
          Eli Collins added a comment -

          Yea, I think moving the ReadaheadPool intialization and shutdown to the DN makes more sense after all. Updated patch attached.

          Show
          Eli Collins added a comment - Yea, I think moving the ReadaheadPool intialization and shutdown to the DN makes more sense after all. Updated patch attached.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539125/hdfs-3754.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2951//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/2951//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2951//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539125/hdfs-3754.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2951//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/2951//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2951//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          Move the resetting of the static instance member to it's own static method (out of the instance method) to appease jenkins.

          Show
          Eli Collins added a comment - Move the resetting of the static instance member to it's own static method (out of the instance method) to appease jenkins.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539170/hdfs-3754.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2957//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2957//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539170/hdfs-3754.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2957//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2957//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          I'm a little confused, doesn't this have the same problem as before for tests? i.e one DN shutting down will call ReadaheadPool.shutdown() on the only instance, but anyone else who keeps a reference to that one will get exceptions trying to submit to the executor?

          Maybe we could fix this more easily by just changing the ReadaheadPool ThreadFactory so that the threads don't get created inside any thread group? Then they'd exit on JVM exit properly.

          Show
          Todd Lipcon added a comment - I'm a little confused, doesn't this have the same problem as before for tests? i.e one DN shutting down will call ReadaheadPool.shutdown() on the only instance, but anyone else who keeps a reference to that one will get exceptions trying to submit to the executor? Maybe we could fix this more easily by just changing the ReadaheadPool ThreadFactory so that the threads don't get created inside any thread group? Then they'd exit on JVM exit properly.
          Hide
          Eli Collins added a comment -

          Good point, it's possible in a test that an existing active block sender could race with the shutdown of another DN and submit to the pool that's been shutdown.
          I like the idea making the ReadaheadPool pool not part of the dataXceiverServer thread group, this can actually be accomplished more easily by just moving the initialization from BlockSender to DataNode, which is a more logical place anyway. Updated patch attached.

          Show
          Eli Collins added a comment - Good point, it's possible in a test that an existing active block sender could race with the shutdown of another DN and submit to the pool that's been shutdown. I like the idea making the ReadaheadPool pool not part of the dataXceiverServer thread group, this can actually be accomplished more easily by just moving the initialization from BlockSender to DataNode, which is a more logical place anyway. Updated patch attached.
          Hide
          Colin Patrick McCabe added a comment -

          Initializing the ReadaheadPool in DataNode seems like a good idea to me. I also tested the latest patch, and it worked for me.

          Would it be worthwhile to add a comment to ReadaheadPool about the importance of having the correct thread context? Or maybe just a reference to this JIRA. I know that I definitely wouldn't have considered the importance of thread context in this situation.

          Show
          Colin Patrick McCabe added a comment - Initializing the ReadaheadPool in DataNode seems like a good idea to me. I also tested the latest patch, and it worked for me. Would it be worthwhile to add a comment to ReadaheadPool about the importance of having the correct thread context? Or maybe just a reference to this JIRA. I know that I definitely wouldn't have considered the importance of thread context in this situation.
          Hide
          Eli Collins added a comment -

          Colin, thanks, sure, I'll add a comment.

          Show
          Eli Collins added a comment - Colin, thanks, sure, I'll add a comment.
          Hide
          Eli Collins added a comment -

          Updated patch with comment per Colin's suggestion.

          Show
          Eli Collins added a comment - Updated patch with comment per Colin's suggestion.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539360/hdfs-3754.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2958//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2958//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539360/hdfs-3754.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2958//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2958//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          +1

          Show
          Todd Lipcon added a comment - +1
          Hide
          Eli Collins added a comment -

          Thanks, here's the patch for branch-1.

          Show
          Eli Collins added a comment - Thanks, here's the patch for branch-1.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539370/hdfs-3754.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestFileConcurrentReader

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2959//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2959//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539370/hdfs-3754.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2959//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2959//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          The specific test case which failed was recently re-enabled after having been disabled for a very long time. Quite possible the failure is unrelated to this particular patch, but worth looking in to.

          Show
          Aaron T. Myers added a comment - The specific test case which failed was recently re-enabled after having been disabled for a very long time. Quite possible the failure is unrelated to this particular patch, but worth looking in to.
          Hide
          Eli Collins added a comment -

          Yea, was looking at that. Don't think it's related filed HDFS-3770 with the rationale. Sanity checked that this test passes for me w/ this patch applied for a couple of runs.

          Show
          Eli Collins added a comment - Yea, was looking at that. Don't think it's related filed HDFS-3770 with the rationale. Sanity checked that this test passes for me w/ this patch applied for a couple of runs.
          Hide
          Eli Collins added a comment -

          I've committed this to trunk and merged to branch-2.

          Show
          Eli Collins added a comment - I've committed this to trunk and merged to branch-2.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2561 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2561/)
          HDFS-3754. BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2561 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2561/ ) HDFS-3754 . BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2626 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2626/)
          HDFS-3754. BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2626 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2626/ ) HDFS-3754 . BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2580 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2580/)
          HDFS-3754. BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2580 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2580/ ) HDFS-3754 . BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Hide
          Eli Collins added a comment -

          Updated branch-1 patch. The fix for branch-1 also requires that the invocation of Executors#defaultThreadFactory be hoisted to the constructor, this is because the ThreadGroup used is that of the thread invoking defaultThreadFactory and we want that to be the primary thread group rather than the thread group of whatever thread causes the pool to create a new thread. Trunk doesn't have this problem because it uses guava's ThreadFactoryBuilder which behaves similarly (does not invoke defaultThreadFactory on each newThread call). Running ant test -Dtestcase=TestSeekBug -Dcompile.native=true no longer hangs with this patch.

          Show
          Eli Collins added a comment - Updated branch-1 patch. The fix for branch-1 also requires that the invocation of Executors#defaultThreadFactory be hoisted to the constructor, this is because the ThreadGroup used is that of the thread invoking defaultThreadFactory and we want that to be the primary thread group rather than the thread group of whatever thread causes the pool to create a new thread. Trunk doesn't have this problem because it uses guava's ThreadFactoryBuilder which behaves similarly (does not invoke defaultThreadFactory on each newThread call). Running ant test -Dtestcase=TestSeekBug -Dcompile.native=true no longer hangs with this patch.
          Hide
          Eli Collins added a comment -

          Forgot to mention, in this case DataXceiverServer is creating the threads and it has the ThreadGroup that the DN is checking the count against which is why we don't want threads created with this ThreadGroup, and logically the ReadaHead thread pool doesn't need to share a group with the dataXceiverServer threads.

          Show
          Eli Collins added a comment - Forgot to mention, in this case DataXceiverServer is creating the threads and it has the ThreadGroup that the DN is checking the count against which is why we don't want threads created with this ThreadGroup, and logically the ReadaHead thread pool doesn't need to share a group with the dataXceiverServer threads.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1129 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1129/)
          HDFS-3754. BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1129 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1129/ ) HDFS-3754 . BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1161 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1161/)
          HDFS-3754. BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1161 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1161/ ) HDFS-3754 . BlockSender doesn't shutdown ReadaheadPool threads. Contributed by Eli Collins (Revision 1370495) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370495 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Hide
          Eli Collins added a comment -

          I've merged this to branch-1.

          Show
          Eli Collins added a comment - I've merged this to branch-1.

            People

            • Assignee:
              Eli Collins
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development