Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12984

Add GenericTestUtils.getTestDir method and use it for temporary directory in tests

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: build, test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We have seen some tests had been used the path test/build/data to store files, so leaking files which fail the new post-build RAT test checks on Jenkins (and dirtying all development systems with paths which mvn clean will miss.
      In order not to occur these bugs such as MAPREDUCE-6589 and HDFS-9571 again, we'd like to introduce new utility methods to get a temporary directory path easily, and use the methods in tests.

      1. HADOOP-12984.branch-2.8.00.patch
        98 kB
        Xiaoyu Yao
      2. HADOOP-12984-003.patch
        133 kB
        Akira Ajisaka
      3. HDFS-9263-001.patch
        46 kB
        Steve Loughran
      4. HDFS-9263-002.patch
        46 kB
        Chris Nauroth
      5. HDFS-9263-003.patch
        133 kB
        Vinayakumar B

        Issue Links

          Activity

          Hide
          jagadesh.kiran Jagadesh Kiran N added a comment -

          Thanks Steve Loughran for reporting the same, i am planning to work on this., if you have already started work on this, please let me know

          Show
          jagadesh.kiran Jagadesh Kiran N added a comment - Thanks Steve Loughran for reporting the same, i am planning to work on this., if you have already started work on this, please let me know
          Hide
          stevel@apache.org Steve Loughran added a comment -

          This covers -common and a couple of other places too, but it's where HDFS did the most

          1. uses of test.build.data have been unified
          2. many buildups of test dirs now use something random, rather than a hard-coded path like "dfs". This includes minidfs cluster...which should improve parallelism on test runs.
          3. I've kept the bits of code that feed the paths into Path constructors independent of the ones that want File instances. Why? Not sure what happens in windows and whether the path stuff really wants forward-slash paths (but then...what happens if maven has set test.build.data to an absolute path in the local FS?)
          Show
          stevel@apache.org Steve Loughran added a comment - This covers -common and a couple of other places too, but it's where HDFS did the most uses of test.build.data have been unified many buildups of test dirs now use something random, rather than a hard-coded path like "dfs". This includes minidfs cluster...which should improve parallelism on test runs. I've kept the bits of code that feed the paths into Path constructors independent of the ones that want File instances. Why? Not sure what happens in windows and whether the path stuff really wants forward-slash paths (but then...what happens if maven has set test.build.data to an absolute path in the local FS?)
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I'd already pulled out the IDE, though it'll take some time hardening —how about you review the code

          Show
          stevel@apache.org Steve Loughran added a comment - I'd already pulled out the IDE, though it'll take some time hardening —how about you review the code
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 12m 3s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 36 new or modified test files.
          +1 javac 8m 54s There were no new javac warning messages.
          +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 2m 24s There were no new checkstyle issues.
          -1 whitespace 0m 6s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 42s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 4m 52s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 9m 0s Tests failed in hadoop-common.
          -1 hdfs tests 22m 59s Tests failed in hadoop-hdfs.
              63m 1s  



          Reason Tests
          Failed unit tests hadoop.fs.TestLocalFsFCStatistics
            hadoop.hdfs.TestParallelShortCircuitRead
            hadoop.fs.contract.hdfs.TestHDFSContractMkdir
            hadoop.hdfs.server.namenode.TestAllowFormat
            hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
            hadoop.hdfs.qjournal.TestMiniJournalCluster
            hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics
            hadoop.cli.TestDeleteCLI
            hadoop.hdfs.tools.TestDFSZKFailoverController
            hadoop.hdfs.TestFileLengthOnClusterRestart
            hadoop.hdfs.TestAppendSnapshotTruncate
            hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
            hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory
            hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
            hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
            hadoop.cli.TestHDFSCLI
            hadoop.hdfs.server.namenode.TestCheckpoint
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.tools.TestGetGroups
            hadoop.hdfs.TestRemoteBlockReader2
            hadoop.hdfs.server.namenode.TestStartup
            hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy
            hadoop.hdfs.TestDFSStorageStateRecovery
            hadoop.hdfs.server.namenode.TestFSImageWithXAttr
            hadoop.hdfs.TestRemoteBlockReader
            hadoop.hdfs.TestMultiThreadedHflush
            hadoop.fs.contract.hdfs.TestHDFSContractRename
            hadoop.hdfs.TestBlockReaderLocal
            hadoop.cli.TestCacheAdminCLI
            hadoop.hdfs.server.mover.TestMover
            hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
            hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
            hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
            hadoop.hdfs.server.namenode.TestNameNodeRecovery
            hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
            hadoop.fs.loadGenerator.TestLoadGenerator
            hadoop.hdfs.server.namenode.TestFSImageWithAcl
            hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
            hadoop.fs.TestFcHdfsSetUMask
            hadoop.hdfs.TestPread
            hadoop.hdfs.server.namenode.TestFSEditLogLoader
            hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation
            hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
            hadoop.hdfs.crypto.TestHdfsCryptoStreams
            hadoop.fs.viewfs.TestViewFsFileStatusHdfs
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl
            hadoop.hdfs.TestDFSAddressConfig
            hadoop.tracing.TestTracingShortCircuitLocalRead
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            hadoop.hdfs.server.namenode.TestFSDirectory
            hadoop.tracing.TestTracing
            hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
            hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
            hadoop.hdfs.TestParallelRead
            hadoop.hdfs.TestDFSStripedInputStream
            hadoop.hdfs.tools.TestDelegationTokenFetcher
            hadoop.hdfs.TestRestartDFS
            hadoop.fs.TestWebHdfsFileContextMainOperations
            hadoop.hdfs.TestParallelShortCircuitReadNoChecksum
            hadoop.hdfs.server.blockmanagement.TestSequentialBlockId
            hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages
            hadoop.hdfs.TestSetTimes
            hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling
            hadoop.hdfs.TestParallelShortCircuitLegacyRead
            hadoop.net.TestNetworkTopology
            hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
            hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
            hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory
            hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
            hadoop.fs.TestFcHdfsCreateMkdir
            hadoop.hdfs.server.namenode.TestDeadDatanode
            hadoop.hdfs.server.namenode.ha.TestNNHealthCheck
            hadoop.hdfs.TestHFlush
            hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
            hadoop.hdfs.TestFetchImage
            hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
            hadoop.hdfs.server.namenode.ha.TestDNFencing
            hadoop.hdfs.TestDFSUpgrade
            hadoop.fs.TestFcHdfsPermission
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks
            hadoop.hdfs.server.namenode.TestProtectedDirectories
            hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            hadoop.hdfs.server.datanode.TestDataNodeExit
            hadoop.hdfs.TestMissingBlocksAlert
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.server.namenode.ha.TestHAMetrics
            hadoop.hdfs.TestQuota
            hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer
            hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot
            hadoop.hdfs.server.namenode.TestQuotaByStorageType
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.server.namenode.TestStorageRestore
            hadoop.hdfs.server.namenode.TestSaveNamespace
            hadoop.hdfs.server.namenode.TestParallelImageWrite
            hadoop.hdfs.tools.TestDebugAdmin
            hadoop.hdfs.server.datanode.TestDiskError
            hadoop.hdfs.TestPersistBlocks
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover
            hadoop.hdfs.server.namenode.TestNameEditsConfigs
            hadoop.hdfs.server.namenode.TestNameNodeRpcServer
            hadoop.hdfs.qjournal.server.TestJournal
            hadoop.hdfs.server.namenode.TestSecureNameNode
            hadoop.hdfs.server.namenode.TestSnapshotPathINodes
            hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
            hadoop.hdfs.TestModTime
            hadoop.hdfs.server.blockmanagement.TestBlockManager
            hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA
            hadoop.hdfs.TestGetBlocks
            hadoop.hdfs.TestDFSStartupVersions
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.TestRenameWhileOpen
            hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy
            hadoop.fs.viewfs.TestViewFsWithXAttrs
            hadoop.cli.TestErasureCodingCLI
            hadoop.tools.TestJMXGet
            hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
            hadoop.hdfs.TestFileAppend4
            hadoop.TestGenericRefresh
            hadoop.hdfs.TestRecoverStripedFile
            hadoop.hdfs.server.namenode.TestAddBlock
            hadoop.hdfs.TestDataTransferKeepalive
            hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.server.namenode.TestINodeFile
            hadoop.hdfs.TestFileAppend2
            hadoop.hdfs.server.datanode.TestDataNodeECN
            hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA
            hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
            hadoop.hdfs.server.namenode.TestAuditLogs
            hadoop.fs.TestSWebHdfsFileContextMainOperations
            hadoop.hdfs.TestInjectionForSimulatedStorage
            hadoop.hdfs.server.namenode.TestFsck
            hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs
            hadoop.fs.contract.hdfs.TestHDFSContractDelete
            hadoop.fs.contract.hdfs.TestHDFSContractSetTimes
            hadoop.hdfs.TestLocalDFS
            hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
            hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
            hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.server.namenode.TestNameNodeAcl
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForXAttr
            hadoop.hdfs.TestConnCache
            hadoop.hdfs.server.namenode.TestStripedINodeFile
            hadoop.hdfs.TestGetFileChecksum
            hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.TestReservedRawPaths
            hadoop.hdfs.TestReadStripedFileWithDecoding
            hadoop.TestRefreshCallQueue
            hadoop.hdfs.TestDisableConnCache
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement
            hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
            hadoop.hdfs.server.blockmanagement.TestBlockReportRateLimiting
            hadoop.cli.TestXAttrCLI
            hadoop.hdfs.server.namenode.TestMetadataVersionOutput
            hadoop.hdfs.TestParallelShortCircuitReadUnCached
            hadoop.fs.TestUrlStreamHandler
            hadoop.hdfs.server.namenode.TestNameNodeMXBean
            hadoop.hdfs.server.namenode.TestValidateConfigurationSettings
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
            hadoop.fs.TestEnhancedByteBufferAccess
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename
            hadoop.hdfs.server.namenode.TestFSNamesystem
            hadoop.hdfs.server.namenode.TestAuditLogger
            hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.server.namenode.TestFSImage
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage
            hadoop.hdfs.server.datanode.TestDataNodeInitStorage
            hadoop.fs.viewfs.TestViewFileSystemWithXAttrs
            hadoop.hdfs.server.namenode.TestMalformedURLs
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.fs.TestHDFSFileContextMainOperations
            hadoop.hdfs.TestSafeMode
            hadoop.hdfs.qjournal.client.TestEpochsAreUnique
            hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
            hadoop.hdfs.server.namenode.TestBackupNode
            hadoop.hdfs.TestDatanodeLayoutUpgrade
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.hdfs.server.namenode.TestFileContextAcl
            hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
            hadoop.hdfs.server.namenode.TestAclConfigFlag
            hadoop.hdfs.TestFileStatusWithECPolicy
            hadoop.hdfs.TestWriteRead
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
            hadoop.hdfs.tools.TestDFSAdminWithHA
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
            hadoop.hdfs.TestExternalBlockReader
            hadoop.hdfs.server.namenode.TestMetaSave
            hadoop.fs.contract.hdfs.TestHDFSContractAppend
            hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure
            hadoop.hdfs.TestDFSStripedOutputStream
            hadoop.hdfs.TestHdfsAdmin
            hadoop.fs.contract.hdfs.TestHDFSContractOpen
            hadoop.hdfs.server.namenode.TestAddStripedBlocks
            hadoop.hdfs.server.namenode.TestGenericJournalConf
            hadoop.hdfs.TestFileCreation
            hadoop.hdfs.server.namenode.snapshot.TestSnapshot
            hadoop.hdfs.server.mover.TestStorageMover
            hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
            hadoop.hdfs.TestClientReportBadBlock
            hadoop.hdfs.server.namenode.TestAddBlockRetry
            hadoop.tracing.TestTraceAdmin
            hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
            hadoop.hdfs.server.namenode.TestBlockUnderConstruction
            hadoop.hdfs.TestFileCorruption
            hadoop.hdfs.server.namenode.TestHDFSConcat
            hadoop.hdfs.server.datanode.TestDataNodeMXBean
            hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation
            hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.TestClose
            hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
            hadoop.hdfs.TestAbandonBlock
            hadoop.hdfs.TestDFSShell
            hadoop.hdfs.server.namenode.ha.TestHAFsck
            hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant
            hadoop.hdfs.TestDFSInputStream
            hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional
            hadoop.hdfs.TestLeaseRecovery
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.tools.TestDFSAdmin
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.TestWriteBlockGetsBlockLengthHint
            hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
            hadoop.hdfs.server.namenode.TestFsLimits
            hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
            hadoop.fs.viewfs.TestViewFsHdfs
            hadoop.fs.contract.hdfs.TestHDFSContractConcat
            hadoop.hdfs.server.namenode.TestFileLimit
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing
            hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
            hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.TestHDFSTrash
            hadoop.hdfs.server.datanode.TestCachingStrategy
            hadoop.hdfs.TestSmallBlock
            hadoop.hdfs.web.TestWebHDFSAcl
            hadoop.cli.TestCryptoAdminCLI
            hadoop.hdfs.TestFileCreationDelete
            hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler
            hadoop.hdfs.tools.TestDFSHAAdminMiniCluster
            hadoop.hdfs.qjournal.server.TestJournalNodeMXBean
            hadoop.hdfs.TestReplication
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            hadoop.cli.TestAclCLI
            hadoop.fs.viewfs.TestViewFileSystemWithAcls
            hadoop.hdfs.server.namenode.TestEditLogAutoroll
            hadoop.fs.TestSymlinkHdfsFileContext
            hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
            hadoop.hdfs.server.namenode.TestTransferFsImage
            hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.server.namenode.TestEditLogRace
            hadoop.fs.TestSymlinkHdfsFileSystem
            hadoop.hdfs.TestFileAppend3
            hadoop.hdfs.server.namenode.TestFileContextXAttr
            hadoop.hdfs.TestMiniDFSCluster
            hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
            hadoop.fs.TestUnbuffer
            hadoop.hdfs.TestPipelines
            hadoop.hdfs.TestErasureCodingPolicies
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.namenode.ha.TestHASafeMode
            hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot
            hadoop.hdfs.server.namenode.TestINodeAttributeProvider
            hadoop.hdfs.qjournal.client.TestQuorumJournalManager
            hadoop.hdfs.qjournal.TestSecureNNWithQJM
            hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
            hadoop.hdfs.TestBlockReaderFactory
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication
            hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.TestFSOutputSummer
            hadoop.hdfs.server.blockmanagement.TestSequentialBlockGroupId
            hadoop.hdfs.TestBlockReaderLocalLegacy
            hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
            hadoop.fs.viewfs.TestViewFsWithAcls
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.server.blockmanagement.TestPendingReplication
            hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
            hadoop.fs.TestResolveHdfsSymlink
            hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
            hadoop.hdfs.server.namenode.TestXAttrConfigFlag
            hadoop.hdfs.TestFileConcurrentReader
            hadoop.hdfs.server.blockmanagement.TestNodeCount
            hadoop.fs.viewfs.TestViewFsDefaultValue
            hadoop.hdfs.TestAppendDifferentChecksum
            hadoop.hdfs.qjournal.server.TestJournalNode
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus
            hadoop.hdfs.server.namenode.TestCreateEditsLog
            hadoop.hdfs.server.namenode.TestFSNamesystemMBean
            hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA
            hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocks
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary
            hadoop.hdfs.web.TestWebHDFSXAttr
            hadoop.hdfs.server.namenode.TestNameNodeXAttr
            hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
            hadoop.hdfs.server.datanode.TestHSync
            hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
            hadoop.fs.viewfs.TestViewFileSystemHdfs
            hadoop.hdfs.server.namenode.TestRecoverStripedBlocks
            hadoop.hdfs.TestParallelUnixDomainRead
            hadoop.hdfs.qjournal.client.TestQJMWithFaults
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.tools.TestStoragePolicyCommands
            hadoop.hdfs.TestDFSRemove
            hadoop.fs.viewfs.TestViewFsAtHdfsRoot
            hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
            hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks
            hadoop.hdfs.server.namenode.TestHostsFiles
            hadoop.hdfs.qjournal.TestNNWithQJM
            hadoop.hdfs.TestFileCreationClient
            hadoop.hdfs.TestRollingUpgradeDowngrade
            hadoop.fs.contract.hdfs.TestHDFSContractCreate
            hadoop.fs.permission.TestStickyBit
            hadoop.hdfs.TestEncryptionZonesWithKMS
            hadoop.fs.TestSymlinkHdfsDisable
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.server.namenode.TestDeleteRace
            hadoop.hdfs.TestLease
            hadoop.hdfs.TestLargeBlock
            hadoop.fs.TestGlobPaths
            hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes
            hadoop.hdfs.server.namenode.TestEditLogJournalFailures
            hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean
            hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.datanode.TestDatanodeProtocolRetryPolicy
            hadoop.hdfs.server.datanode.TestDataNodeMetricsLogger
            hadoop.hdfs.server.namenode.TestSecondaryWebUi
            hadoop.fs.contract.hdfs.TestHDFSContractSeek
            hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
            hadoop.fs.shell.TestHdfsTextCommand
            hadoop.hdfs.server.namenode.snapshot.TestGetContentSummaryWithSnapshot
            hadoop.hdfs.TestBlockMissingException
          Timed out tests org.apache.hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767377/HDFS-9263-001.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 151eab2
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13050/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13050/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/13050/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13050/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13050/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13050/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 12m 3s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 36 new or modified test files. +1 javac 8m 54s There were no new javac warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 24s There were no new checkstyle issues. -1 whitespace 0m 6s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 42s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 52s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 9m 0s Tests failed in hadoop-common. -1 hdfs tests 22m 59s Tests failed in hadoop-hdfs.     63m 1s   Reason Tests Failed unit tests hadoop.fs.TestLocalFsFCStatistics   hadoop.hdfs.TestParallelShortCircuitRead   hadoop.fs.contract.hdfs.TestHDFSContractMkdir   hadoop.hdfs.server.namenode.TestAllowFormat   hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens   hadoop.hdfs.qjournal.TestMiniJournalCluster   hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics   hadoop.cli.TestDeleteCLI   hadoop.hdfs.tools.TestDFSZKFailoverController   hadoop.hdfs.TestFileLengthOnClusterRestart   hadoop.hdfs.TestAppendSnapshotTruncate   hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing   hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory   hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate   hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.namenode.TestCheckpoint   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.tools.TestGetGroups   hadoop.hdfs.TestRemoteBlockReader2   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy   hadoop.hdfs.TestDFSStorageStateRecovery   hadoop.hdfs.server.namenode.TestFSImageWithXAttr   hadoop.hdfs.TestRemoteBlockReader   hadoop.hdfs.TestMultiThreadedHflush   hadoop.fs.contract.hdfs.TestHDFSContractRename   hadoop.hdfs.TestBlockReaderLocal   hadoop.cli.TestCacheAdminCLI   hadoop.hdfs.server.mover.TestMover   hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks   hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality   hadoop.hdfs.server.namenode.TestNameNodeRecovery   hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.fs.loadGenerator.TestLoadGenerator   hadoop.hdfs.server.namenode.TestFSImageWithAcl   hadoop.hdfs.server.namenode.TestLargeDirectoryDelete   hadoop.fs.TestFcHdfsSetUMask   hadoop.hdfs.TestPread   hadoop.hdfs.server.namenode.TestFSEditLogLoader   hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation   hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   hadoop.hdfs.crypto.TestHdfsCryptoStreams   hadoop.fs.viewfs.TestViewFsFileStatusHdfs   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl   hadoop.hdfs.TestDFSAddressConfig   hadoop.tracing.TestTracingShortCircuitLocalRead   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.server.namenode.TestFSDirectory   hadoop.tracing.TestTracing   hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol   hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles   hadoop.hdfs.TestParallelRead   hadoop.hdfs.TestDFSStripedInputStream   hadoop.hdfs.tools.TestDelegationTokenFetcher   hadoop.hdfs.TestRestartDFS   hadoop.fs.TestWebHdfsFileContextMainOperations   hadoop.hdfs.TestParallelShortCircuitReadNoChecksum   hadoop.hdfs.server.blockmanagement.TestSequentialBlockId   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages   hadoop.hdfs.TestSetTimes   hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling   hadoop.hdfs.TestParallelShortCircuitLegacyRead   hadoop.net.TestNetworkTopology   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS   hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory   hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   hadoop.fs.TestFcHdfsCreateMkdir   hadoop.hdfs.server.namenode.TestDeadDatanode   hadoop.hdfs.server.namenode.ha.TestNNHealthCheck   hadoop.hdfs.TestHFlush   hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled   hadoop.hdfs.TestFetchImage   hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication   hadoop.hdfs.server.namenode.ha.TestDNFencing   hadoop.hdfs.TestDFSUpgrade   hadoop.fs.TestFcHdfsPermission   hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks   hadoop.hdfs.server.namenode.TestProtectedDirectories   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.server.datanode.TestDataNodeExit   hadoop.hdfs.TestMissingBlocksAlert   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.server.namenode.ha.TestHAMetrics   hadoop.hdfs.TestQuota   hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer   hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot   hadoop.hdfs.server.namenode.TestQuotaByStorageType   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.namenode.TestStorageRestore   hadoop.hdfs.server.namenode.TestSaveNamespace   hadoop.hdfs.server.namenode.TestParallelImageWrite   hadoop.hdfs.tools.TestDebugAdmin   hadoop.hdfs.server.datanode.TestDiskError   hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover   hadoop.hdfs.server.namenode.TestNameEditsConfigs   hadoop.hdfs.server.namenode.TestNameNodeRpcServer   hadoop.hdfs.qjournal.server.TestJournal   hadoop.hdfs.server.namenode.TestSecureNameNode   hadoop.hdfs.server.namenode.TestSnapshotPathINodes   hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer   hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters   hadoop.hdfs.TestModTime   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA   hadoop.hdfs.TestGetBlocks   hadoop.hdfs.TestDFSStartupVersions   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.TestRenameWhileOpen   hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy   hadoop.fs.viewfs.TestViewFsWithXAttrs   hadoop.cli.TestErasureCodingCLI   hadoop.tools.TestJMXGet   hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead   hadoop.hdfs.TestFileAppend4   hadoop.TestGenericRefresh   hadoop.hdfs.TestRecoverStripedFile   hadoop.hdfs.server.namenode.TestAddBlock   hadoop.hdfs.TestDataTransferKeepalive   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.server.namenode.TestINodeFile   hadoop.hdfs.TestFileAppend2   hadoop.hdfs.server.datanode.TestDataNodeECN   hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA   hadoop.hdfs.server.namenode.TestProcessCorruptBlocks   hadoop.hdfs.server.namenode.TestAuditLogs   hadoop.fs.TestSWebHdfsFileContextMainOperations   hadoop.hdfs.TestInjectionForSimulatedStorage   hadoop.hdfs.server.namenode.TestFsck   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs   hadoop.fs.contract.hdfs.TestHDFSContractDelete   hadoop.fs.contract.hdfs.TestHDFSContractSetTimes   hadoop.hdfs.TestLocalDFS   hadoop.hdfs.server.namenode.TestListCorruptFileBlocks   hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.server.namenode.TestNameNodeAcl   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForXAttr   hadoop.hdfs.TestConnCache   hadoop.hdfs.server.namenode.TestStripedINodeFile   hadoop.hdfs.TestGetFileChecksum   hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestReservedRawPaths   hadoop.hdfs.TestReadStripedFileWithDecoding   hadoop.TestRefreshCallQueue   hadoop.hdfs.TestDisableConnCache   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   hadoop.hdfs.server.blockmanagement.TestBlockReportRateLimiting   hadoop.cli.TestXAttrCLI   hadoop.hdfs.server.namenode.TestMetadataVersionOutput   hadoop.hdfs.TestParallelShortCircuitReadUnCached   hadoop.fs.TestUrlStreamHandler   hadoop.hdfs.server.namenode.TestNameNodeMXBean   hadoop.hdfs.server.namenode.TestValidateConfigurationSettings   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot   hadoop.fs.TestEnhancedByteBufferAccess   hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename   hadoop.hdfs.server.namenode.TestFSNamesystem   hadoop.hdfs.server.namenode.TestAuditLogger   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.namenode.TestFSImage   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage   hadoop.hdfs.server.datanode.TestDataNodeInitStorage   hadoop.fs.viewfs.TestViewFileSystemWithXAttrs   hadoop.hdfs.server.namenode.TestMalformedURLs   hadoop.hdfs.server.namenode.TestEditLog   hadoop.fs.TestHDFSFileContextMainOperations   hadoop.hdfs.TestSafeMode   hadoop.hdfs.qjournal.client.TestEpochsAreUnique   hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.hdfs.TestDatanodeLayoutUpgrade   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.server.namenode.TestFileContextAcl   hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement   hadoop.hdfs.server.namenode.TestAclConfigFlag   hadoop.hdfs.TestFileStatusWithECPolicy   hadoop.hdfs.TestWriteRead   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport   hadoop.hdfs.TestExternalBlockReader   hadoop.hdfs.server.namenode.TestMetaSave   hadoop.fs.contract.hdfs.TestHDFSContractAppend   hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure   hadoop.hdfs.TestDFSStripedOutputStream   hadoop.hdfs.TestHdfsAdmin   hadoop.fs.contract.hdfs.TestHDFSContractOpen   hadoop.hdfs.server.namenode.TestAddStripedBlocks   hadoop.hdfs.server.namenode.TestGenericJournalConf   hadoop.hdfs.TestFileCreation   hadoop.hdfs.server.namenode.snapshot.TestSnapshot   hadoop.hdfs.server.mover.TestStorageMover   hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot   hadoop.hdfs.TestClientReportBadBlock   hadoop.hdfs.server.namenode.TestAddBlockRetry   hadoop.tracing.TestTraceAdmin   hadoop.hdfs.server.namenode.TestSecurityTokenEditLog   hadoop.hdfs.server.namenode.TestBlockUnderConstruction   hadoop.hdfs.TestFileCorruption   hadoop.hdfs.server.namenode.TestHDFSConcat   hadoop.hdfs.server.datanode.TestDataNodeMXBean   hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.TestClose   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics   hadoop.hdfs.TestAbandonBlock   hadoop.hdfs.TestDFSShell   hadoop.hdfs.server.namenode.ha.TestHAFsck   hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant   hadoop.hdfs.TestDFSInputStream   hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional   hadoop.hdfs.TestLeaseRecovery   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.tools.TestDFSAdmin   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.TestWriteBlockGetsBlockLengthHint   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.namenode.TestFsLimits   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   hadoop.fs.viewfs.TestViewFsHdfs   hadoop.fs.contract.hdfs.TestHDFSContractConcat   hadoop.hdfs.server.namenode.TestFileLimit   hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing   hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.TestHDFSTrash   hadoop.hdfs.server.datanode.TestCachingStrategy   hadoop.hdfs.TestSmallBlock   hadoop.hdfs.web.TestWebHDFSAcl   hadoop.cli.TestCryptoAdminCLI   hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler   hadoop.hdfs.tools.TestDFSHAAdminMiniCluster   hadoop.hdfs.qjournal.server.TestJournalNodeMXBean   hadoop.hdfs.TestReplication   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   hadoop.cli.TestAclCLI   hadoop.fs.viewfs.TestViewFileSystemWithAcls   hadoop.hdfs.server.namenode.TestEditLogAutoroll   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.hdfs.server.namenode.TestNameNodeResourceChecker   hadoop.hdfs.server.namenode.TestTransferFsImage   hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.server.namenode.TestEditLogRace   hadoop.fs.TestSymlinkHdfsFileSystem   hadoop.hdfs.TestFileAppend3   hadoop.hdfs.server.namenode.TestFileContextXAttr   hadoop.hdfs.TestMiniDFSCluster   hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean   hadoop.fs.TestUnbuffer   hadoop.hdfs.TestPipelines   hadoop.hdfs.TestErasureCodingPolicies   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot   hadoop.hdfs.server.namenode.TestINodeAttributeProvider   hadoop.hdfs.qjournal.client.TestQuorumJournalManager   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps   hadoop.hdfs.TestBlockReaderFactory   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestFSOutputSummer   hadoop.hdfs.server.blockmanagement.TestSequentialBlockGroupId   hadoop.hdfs.TestBlockReaderLocalLegacy   hadoop.hdfs.server.namenode.TestFSImageWithSnapshot   hadoop.fs.viewfs.TestViewFsWithAcls   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.blockmanagement.TestPendingReplication   hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots   hadoop.fs.TestResolveHdfsSymlink   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks   hadoop.hdfs.server.namenode.TestXAttrConfigFlag   hadoop.hdfs.TestFileConcurrentReader   hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.fs.viewfs.TestViewFsDefaultValue   hadoop.hdfs.TestAppendDifferentChecksum   hadoop.hdfs.qjournal.server.TestJournalNode   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus   hadoop.hdfs.server.namenode.TestCreateEditsLog   hadoop.hdfs.server.namenode.TestFSNamesystemMBean   hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA   hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocks   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary   hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.server.namenode.TestNameNodeXAttr   hadoop.hdfs.server.blockmanagement.TestReplicationPolicy   hadoop.hdfs.server.datanode.TestHSync   hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer   hadoop.fs.viewfs.TestViewFileSystemHdfs   hadoop.hdfs.server.namenode.TestRecoverStripedBlocks   hadoop.hdfs.TestParallelUnixDomainRead   hadoop.hdfs.qjournal.client.TestQJMWithFaults   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.tools.TestStoragePolicyCommands   hadoop.hdfs.TestDFSRemove   hadoop.fs.viewfs.TestViewFsAtHdfsRoot   hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork   hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks   hadoop.hdfs.server.namenode.TestHostsFiles   hadoop.hdfs.qjournal.TestNNWithQJM   hadoop.hdfs.TestFileCreationClient   hadoop.hdfs.TestRollingUpgradeDowngrade   hadoop.fs.contract.hdfs.TestHDFSContractCreate   hadoop.fs.permission.TestStickyBit   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.fs.TestSymlinkHdfsDisable   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.TestLease   hadoop.hdfs.TestLargeBlock   hadoop.fs.TestGlobPaths   hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes   hadoop.hdfs.server.namenode.TestEditLogJournalFailures   hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean   hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.datanode.TestDatanodeProtocolRetryPolicy   hadoop.hdfs.server.datanode.TestDataNodeMetricsLogger   hadoop.hdfs.server.namenode.TestSecondaryWebUi   hadoop.fs.contract.hdfs.TestHDFSContractSeek   hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart   hadoop.fs.shell.TestHdfsTextCommand   hadoop.hdfs.server.namenode.snapshot.TestGetContentSummaryWithSnapshot   hadoop.hdfs.TestBlockMissingException Timed out tests org.apache.hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767377/HDFS-9263-001.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 151eab2 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13050/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13050/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/13050/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13050/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13050/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13050/console This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          I retriggered the build on this, some of these seem unrelated. Running locally I did see some errors in the changed tests, so will likely need a rev besides this.

          Show
          andrew.wang Andrew Wang added a comment - I retriggered the build on this, some of these seem unrelated. Running locally I did see some errors in the changed tests, so will likely need a rev besides this.
          Hide
          andrew.wang Andrew Wang added a comment -

          Other sort of related comment is that we have all these temp dirs defined in hadoop-project-dist/pom.xml:

              <hadoop.tmp.dir>${project.build.directory}/test</hadoop.tmp.dir>
              <test.build.data>${project.build.directory}/test/data</test.build.data>
              <hadoop.log.dir>${project.build.directory}/log</hadoop.log.dir>
              <test.build.webapps>${project.build.directory}/test-classes/webapps</test.build.webapps>
              <test.cache.data>${project.build.directory}/test-classes</test.cache.data>
              <test.build.classes>${project.build.directory}/test-classes</test.build.classes>
          

          Some of these are dupes, it'd be nice to get rid of whichever ones are less popular.

          Show
          andrew.wang Andrew Wang added a comment - Other sort of related comment is that we have all these temp dirs defined in hadoop-project-dist/pom.xml: <hadoop.tmp.dir>${project.build.directory}/test</hadoop.tmp.dir> <test.build.data>${project.build.directory}/test/data</test.build.data> <hadoop.log.dir>${project.build.directory}/log</hadoop.log.dir> <test.build.webapps>${project.build.directory}/test-classes/webapps</test.build.webapps> <test.cache.data>${project.build.directory}/test-classes</test.cache.data> <test.build.classes>${project.build.directory}/test-classes</test.build.classes> Some of these are dupes, it'd be nice to get rid of whichever ones are less popular.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 9m 40s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 36 new or modified test files.
          +1 javac 7m 53s There were no new javac warning messages.
          +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 2m 16s There were no new checkstyle issues.
          -1 whitespace 0m 6s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 4m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 common tests 6m 56s Tests passed in hadoop-common.
          -1 hdfs tests 30m 18s Tests failed in hadoop-hdfs.
              64m 10s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
            hadoop.fs.TestUrlStreamHandler
            hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
            hadoop.hdfs.server.namenode.TestFileLimit
            hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot
            hadoop.hdfs.server.namenode.TestEditLogAutoroll
            hadoop.TestRefreshCallQueue
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.cli.TestCryptoAdminCLI
            hadoop.fs.viewfs.TestViewFsWithAcls
            hadoop.hdfs.server.namenode.TestAddStripedBlocks
            hadoop.hdfs.server.namenode.TestFSEditLogLoader
            hadoop.hdfs.server.namenode.TestHostsFiles
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy
            hadoop.fs.contract.hdfs.TestHDFSContractDelete
            hadoop.hdfs.server.namenode.TestFileContextAcl
            hadoop.fs.TestFcHdfsSetUMask
            hadoop.fs.TestUnbuffer
            hadoop.hdfs.server.namenode.TestDeleteRace
            hadoop.hdfs.server.namenode.TestFSDirectory
            hadoop.fs.contract.hdfs.TestHDFSContractOpen
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.fs.contract.hdfs.TestHDFSContractMkdir
            hadoop.fs.contract.hdfs.TestHDFSContractAppend
            hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
            hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA
            hadoop.hdfs.server.namenode.TestSecondaryWebUi
            hadoop.hdfs.server.namenode.TestMalformedURLs
            hadoop.hdfs.server.namenode.TestAuditLogger
            hadoop.hdfs.server.namenode.TestRecoverStripedBlocks
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
            hadoop.hdfs.TestDatanodeLayoutUpgrade
            hadoop.hdfs.server.namenode.TestHDFSConcat
            hadoop.hdfs.server.datanode.TestCachingStrategy
            hadoop.hdfs.server.namenode.TestAddBlockRetry
            hadoop.fs.TestSymlinkHdfsFileSystem
            hadoop.fs.viewfs.TestViewFsDefaultValue
            hadoop.fs.TestSymlinkHdfsFileContext
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.TestFSInputChecker
            hadoop.hdfs.web.TestWebHDFSXAttr
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            hadoop.hdfs.server.mover.TestStorageMover
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory
            hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
            hadoop.cli.TestAclCLI
            hadoop.hdfs.server.namenode.ha.TestHAMetrics
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
            hadoop.hdfs.server.namenode.TestFsLimits
            hadoop.hdfs.TestEncryptedTransfer
            hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
            hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional
            hadoop.hdfs.server.datanode.TestHSync
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter
            hadoop.hdfs.TestBlockMissingException
            hadoop.hdfs.server.namenode.TestNameNodeRpcServer
            hadoop.fs.contract.hdfs.TestHDFSContractSeek
            hadoop.hdfs.web.TestWebHDFSAcl
            hadoop.hdfs.server.namenode.TestFileContextXAttr
            hadoop.hdfs.server.namenode.TestAclConfigFlag
            hadoop.hdfs.server.namenode.TestFSImageWithXAttr
            hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
            hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
            hadoop.hdfs.server.namenode.TestFSImageWithAcl
            hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocks
            hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
            hadoop.tracing.TestTracing
            hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
            hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir
            hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.server.namenode.TestParallelImageWrite
            hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
            hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
            hadoop.hdfs.server.namenode.TestGenericJournalConf
            hadoop.fs.viewfs.TestViewFsWithXAttrs
            hadoop.cli.TestErasureCodingCLI
            hadoop.hdfs.server.namenode.snapshot.TestGetContentSummaryWithSnapshot
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.server.namenode.TestEditLogJournalFailures
            hadoop.hdfs.TestPersistBlocks
            hadoop.hdfs.TestFileCreationDelete
            hadoop.hdfs.TestDatanodeReport
            hadoop.tools.TestJMXGet
            hadoop.hdfs.server.namenode.TestNameNodeMXBean
            hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
            hadoop.fs.contract.hdfs.TestHDFSContractCreate
            hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
            hadoop.hdfs.server.namenode.TestCreateEditsLog
            hadoop.hdfs.server.namenode.TestFSNamesystemMBean
            hadoop.hdfs.TestWriteRead
            hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.fs.TestEnhancedByteBufferAccess
            hadoop.fs.TestFcHdfsPermission
            hadoop.hdfs.server.namenode.TestEditLogRace
            hadoop.security.TestRefreshUserMappings
            hadoop.hdfs.TestSafeMode
            hadoop.hdfs.server.namenode.TestAllowFormat
            hadoop.fs.viewfs.TestViewFsHdfs
            hadoop.fs.TestResolveHdfsSymlink
            hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
            hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
            hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica
            hadoop.tracing.TestTracingShortCircuitLocalRead
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus
            hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
            hadoop.hdfs.server.namenode.TestProtectedDirectories
            hadoop.hdfs.qjournal.TestMiniJournalCluster
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.tracing.TestTraceAdmin
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.server.datanode.TestTriggerBlockReport
            hadoop.hdfs.server.namenode.TestINodeFile
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs
            hadoop.hdfs.TestSetTimes
            hadoop.hdfs.server.namenode.TestQuotaByStorageType
            hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer
            hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            hadoop.fs.TestWebHdfsFileContextMainOperations
            hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.fs.TestFcHdfsCreateMkdir
            hadoop.hdfs.server.datanode.TestBPOfferService
            hadoop.hdfs.server.namenode.ha.TestNNHealthCheck
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
            hadoop.hdfs.server.namenode.TestBlockUnderConstruction
            hadoop.hdfs.server.mover.TestMover
            hadoop.hdfs.server.namenode.TestDeadDatanode
            hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy
            hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
            hadoop.hdfs.server.datanode.TestIncrementalBlockReports
            hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
            hadoop.hdfs.server.namenode.TestStorageRestore
            hadoop.hdfs.server.namenode.TestFSNamesystem
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.fs.viewfs.TestViewFileSystemWithXAttrs
            hadoop.hdfs.server.namenode.ha.TestHAFsck
            hadoop.fs.viewfs.TestViewFileSystemWithAcls
            hadoop.hdfs.server.namenode.TestXAttrConfigFlag
            hadoop.hdfs.TestRestartDFS
            hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
            hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot
            hadoop.hdfs.TestInjectionForSimulatedStorage
            hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
            hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys
            hadoop.hdfs.server.datanode.TestDataNodeMXBean
            hadoop.hdfs.server.datanode.TestDataNodeMetrics
            hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes
            hadoop.hdfs.server.namenode.TestSnapshotPathINodes
            hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure
            hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
            hadoop.security.TestPermission
            hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
            hadoop.fs.contract.hdfs.TestHDFSContractConcat
            hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
            hadoop.hdfs.server.namenode.TestFsck
            hadoop.hdfs.server.namenode.TestAuditLogs
            hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
            hadoop.hdfs.server.datanode.TestDataNodeECN
            hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
            hadoop.security.TestPermissionSymlinks
            hadoop.hdfs.server.namenode.TestValidateConfigurationSettings
            hadoop.hdfs.TestRenameWhileOpen
            hadoop.net.TestNetworkTopology
            hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks
            hadoop.cli.TestXAttrCLI
            hadoop.hdfs.server.namenode.TestNameEditsConfigs
            hadoop.hdfs.TestEncryptionZonesWithHA
            hadoop.hdfs.TestMiniDFSCluster
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.qjournal.server.TestJournalNodeMXBean
            hadoop.fs.contract.hdfs.TestHDFSContractRename
            hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
            hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            hadoop.fs.permission.TestStickyBit
            hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
            hadoop.hdfs.TestFileCreation
            hadoop.hdfs.server.namenode.ha.TestHASafeMode
            hadoop.hdfs.server.namenode.TestSecureNameNode
            hadoop.hdfs.server.namenode.snapshot.TestSnapshot
            hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
            hadoop.hdfs.server.namenode.TestTransferFsImage
            hadoop.hdfs.server.namenode.TestStartup
            hadoop.hdfs.server.namenode.TestNameNodeRecovery
            hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover
            hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
            hadoop.fs.contract.hdfs.TestHDFSContractSetTimes
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA
            hadoop.hdfs.server.namenode.TestINodeAttributeProvider
            hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
            hadoop.hdfs.TestDFSShell
            hadoop.hdfs.TestPipelines
            hadoop.fs.loadGenerator.TestLoadGenerator
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename
            hadoop.hdfs.server.namenode.TestStripedINodeFile
            hadoop.cli.TestCacheAdminCLI
            hadoop.fs.TestSWebHdfsFileContextMainOperations
            hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
            hadoop.hdfs.server.namenode.TestMetadataVersionOutput
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation
            hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler
            hadoop.fs.TestHDFSFileContextMainOperations
            hadoop.hdfs.server.namenode.TestSaveNamespace
            hadoop.fs.TestGlobPaths
            hadoop.hdfs.TestRecoverStripedFile
            hadoop.fs.viewfs.TestViewFsFileStatusHdfs
            hadoop.hdfs.server.namenode.TestMetaSave
            hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
            hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA
            hadoop.fs.viewfs.TestViewFsAtHdfsRoot
            hadoop.TestGenericRefresh
            hadoop.hdfs.server.datanode.TestIncrementalBrVariations
            hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.hdfs.server.namenode.TestBackupNode
            hadoop.hdfs.server.namenode.TestNameNodeAcl
            hadoop.fs.shell.TestHdfsTextCommand
            hadoop.cli.TestDeleteCLI
            hadoop.cli.TestHDFSCLI
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
            hadoop.fs.TestSymlinkHdfsDisable
            hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
            hadoop.hdfs.server.namenode.TestFSImage
            hadoop.fs.viewfs.TestViewFileSystemHdfs
            hadoop.hdfs.server.namenode.TestAddBlock
            hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.TestDFSInotifyEventInputStream
            hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
            hadoop.hdfs.server.namenode.TestNameNodeXAttr
            hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold
            hadoop.hdfs.server.namenode.TestCheckpoint
            hadoop.hdfs.TestBlockReaderLocal
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean
            hadoop.hdfs.server.namenode.ha.TestDNFencing
            hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
          Timed out tests org.apache.hadoop.hdfs.TestReplication



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767377/HDFS-9263-001.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 8175c4f
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13059/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13059/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/13059/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13059/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13059/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13059/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 9m 40s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 36 new or modified test files. +1 javac 7m 53s There were no new javac warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 16s There were no new checkstyle issues. -1 whitespace 0m 6s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 6m 56s Tests passed in hadoop-common. -1 hdfs tests 30m 18s Tests failed in hadoop-hdfs.     64m 10s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots   hadoop.fs.TestUrlStreamHandler   hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead   hadoop.hdfs.server.namenode.TestFileLimit   hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot   hadoop.hdfs.server.namenode.TestEditLogAutoroll   hadoop.TestRefreshCallQueue   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.cli.TestCryptoAdminCLI   hadoop.fs.viewfs.TestViewFsWithAcls   hadoop.hdfs.server.namenode.TestAddStripedBlocks   hadoop.hdfs.server.namenode.TestFSEditLogLoader   hadoop.hdfs.server.namenode.TestHostsFiles   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy   hadoop.fs.contract.hdfs.TestHDFSContractDelete   hadoop.hdfs.server.namenode.TestFileContextAcl   hadoop.fs.TestFcHdfsSetUMask   hadoop.fs.TestUnbuffer   hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.server.namenode.TestFSDirectory   hadoop.fs.contract.hdfs.TestHDFSContractOpen   hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.fs.contract.hdfs.TestHDFSContractMkdir   hadoop.fs.contract.hdfs.TestHDFSContractAppend   hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA   hadoop.hdfs.server.namenode.TestSecondaryWebUi   hadoop.hdfs.server.namenode.TestMalformedURLs   hadoop.hdfs.server.namenode.TestAuditLogger   hadoop.hdfs.server.namenode.TestRecoverStripedBlocks   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles   hadoop.hdfs.TestDatanodeLayoutUpgrade   hadoop.hdfs.server.namenode.TestHDFSConcat   hadoop.hdfs.server.datanode.TestCachingStrategy   hadoop.hdfs.server.namenode.TestAddBlockRetry   hadoop.fs.TestSymlinkHdfsFileSystem   hadoop.fs.viewfs.TestViewFsDefaultValue   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestFSInputChecker   hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.server.mover.TestStorageMover   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory   hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol   hadoop.cli.TestAclCLI   hadoop.hdfs.server.namenode.ha.TestHAMetrics   hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap   hadoop.hdfs.server.namenode.TestFsLimits   hadoop.hdfs.TestEncryptedTransfer   hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks   hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional   hadoop.hdfs.server.datanode.TestHSync   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter   hadoop.hdfs.TestBlockMissingException   hadoop.hdfs.server.namenode.TestNameNodeRpcServer   hadoop.fs.contract.hdfs.TestHDFSContractSeek   hadoop.hdfs.web.TestWebHDFSAcl   hadoop.hdfs.server.namenode.TestFileContextXAttr   hadoop.hdfs.server.namenode.TestAclConfigFlag   hadoop.hdfs.server.namenode.TestFSImageWithXAttr   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics   hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot   hadoop.hdfs.server.namenode.TestFSImageWithAcl   hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocks   hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement   hadoop.tracing.TestTracing   hadoop.hdfs.server.namenode.TestListCorruptFileBlocks   hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.server.namenode.TestParallelImageWrite   hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot   hadoop.hdfs.server.namenode.TestNameNodeResourceChecker   hadoop.hdfs.server.namenode.TestGenericJournalConf   hadoop.fs.viewfs.TestViewFsWithXAttrs   hadoop.cli.TestErasureCodingCLI   hadoop.hdfs.server.namenode.snapshot.TestGetContentSummaryWithSnapshot   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.namenode.TestEditLogJournalFailures   hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.TestDatanodeReport   hadoop.tools.TestJMXGet   hadoop.hdfs.server.namenode.TestNameNodeMXBean   hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport   hadoop.fs.contract.hdfs.TestHDFSContractCreate   hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart   hadoop.hdfs.server.namenode.TestCreateEditsLog   hadoop.hdfs.server.namenode.TestFSNamesystemMBean   hadoop.hdfs.TestWriteRead   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.fs.TestEnhancedByteBufferAccess   hadoop.fs.TestFcHdfsPermission   hadoop.hdfs.server.namenode.TestEditLogRace   hadoop.security.TestRefreshUserMappings   hadoop.hdfs.TestSafeMode   hadoop.hdfs.server.namenode.TestAllowFormat   hadoop.fs.viewfs.TestViewFsHdfs   hadoop.fs.TestResolveHdfsSymlink   hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps   hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality   hadoop.hdfs.server.datanode.fsdataset.impl.TestWriteToReplica   hadoop.tracing.TestTracingShortCircuitLocalRead   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus   hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots   hadoop.hdfs.server.namenode.TestProtectedDirectories   hadoop.hdfs.qjournal.TestMiniJournalCluster   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.tracing.TestTraceAdmin   hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.server.datanode.TestTriggerBlockReport   hadoop.hdfs.server.namenode.TestINodeFile   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs   hadoop.hdfs.TestSetTimes   hadoop.hdfs.server.namenode.TestQuotaByStorageType   hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.fs.TestWebHdfsFileContextMainOperations   hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA   hadoop.hdfs.server.balancer.TestBalancer   hadoop.fs.TestFcHdfsCreateMkdir   hadoop.hdfs.server.datanode.TestBPOfferService   hadoop.hdfs.server.namenode.ha.TestNNHealthCheck   hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication   hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters   hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   hadoop.hdfs.server.namenode.TestBlockUnderConstruction   hadoop.hdfs.server.mover.TestMover   hadoop.hdfs.server.namenode.TestDeadDatanode   hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy   hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   hadoop.hdfs.server.datanode.TestIncrementalBlockReports   hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled   hadoop.hdfs.server.namenode.TestStorageRestore   hadoop.hdfs.server.namenode.TestFSNamesystem   hadoop.hdfs.server.namenode.TestEditLog   hadoop.fs.viewfs.TestViewFileSystemWithXAttrs   hadoop.hdfs.server.namenode.ha.TestHAFsck   hadoop.fs.viewfs.TestViewFileSystemWithAcls   hadoop.hdfs.server.namenode.TestXAttrConfigFlag   hadoop.hdfs.TestRestartDFS   hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot   hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot   hadoop.hdfs.TestInjectionForSimulatedStorage   hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes   hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys   hadoop.hdfs.server.datanode.TestDataNodeMXBean   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes   hadoop.hdfs.server.namenode.TestSnapshotPathINodes   hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.security.TestPermission   hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup   hadoop.fs.contract.hdfs.TestHDFSContractConcat   hadoop.hdfs.server.namenode.TestLargeDirectoryDelete   hadoop.hdfs.server.namenode.TestFsck   hadoop.hdfs.server.namenode.TestAuditLogs   hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots   hadoop.hdfs.server.datanode.TestDataNodeECN   hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade   hadoop.security.TestPermissionSymlinks   hadoop.hdfs.server.namenode.TestValidateConfigurationSettings   hadoop.hdfs.TestRenameWhileOpen   hadoop.net.TestNetworkTopology   hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks   hadoop.cli.TestXAttrCLI   hadoop.hdfs.server.namenode.TestNameEditsConfigs   hadoop.hdfs.TestEncryptionZonesWithHA   hadoop.hdfs.TestMiniDFSCluster   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.qjournal.server.TestJournalNodeMXBean   hadoop.fs.contract.hdfs.TestHDFSContractRename   hadoop.hdfs.server.namenode.TestSecurityTokenEditLog   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   hadoop.fs.permission.TestStickyBit   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   hadoop.hdfs.TestFileCreation   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.server.namenode.TestSecureNameNode   hadoop.hdfs.server.namenode.snapshot.TestSnapshot   hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot   hadoop.hdfs.server.namenode.TestTransferFsImage   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.server.namenode.TestNameNodeRecovery   hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate   hadoop.fs.contract.hdfs.TestHDFSContractSetTimes   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA   hadoop.hdfs.server.namenode.TestINodeAttributeProvider   hadoop.hdfs.server.namenode.TestFSImageWithSnapshot   hadoop.hdfs.TestDFSShell   hadoop.hdfs.TestPipelines   hadoop.fs.loadGenerator.TestLoadGenerator   hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename   hadoop.hdfs.server.namenode.TestStripedINodeFile   hadoop.cli.TestCacheAdminCLI   hadoop.fs.TestSWebHdfsFileContextMainOperations   hadoop.hdfs.server.namenode.TestProcessCorruptBlocks   hadoop.hdfs.server.namenode.TestMetadataVersionOutput   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation   hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler   hadoop.fs.TestHDFSFileContextMainOperations   hadoop.hdfs.server.namenode.TestSaveNamespace   hadoop.fs.TestGlobPaths   hadoop.hdfs.TestRecoverStripedFile   hadoop.fs.viewfs.TestViewFsFileStatusHdfs   hadoop.hdfs.server.namenode.TestMetaSave   hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens   hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA   hadoop.fs.viewfs.TestViewFsAtHdfsRoot   hadoop.TestGenericRefresh   hadoop.hdfs.server.datanode.TestIncrementalBrVariations   hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.hdfs.server.namenode.TestNameNodeAcl   hadoop.fs.shell.TestHdfsTextCommand   hadoop.cli.TestDeleteCLI   hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd   hadoop.fs.TestSymlinkHdfsDisable   hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   hadoop.hdfs.server.namenode.TestFSImage   hadoop.fs.viewfs.TestViewFileSystemHdfs   hadoop.hdfs.server.namenode.TestAddBlock   hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.TestDFSInotifyEventInputStream   hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot   hadoop.hdfs.server.namenode.TestNameNodeXAttr   hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold   hadoop.hdfs.server.namenode.TestCheckpoint   hadoop.hdfs.TestBlockReaderLocal   hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean   hadoop.hdfs.server.namenode.ha.TestDNFencing   hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade Timed out tests org.apache.hadoop.hdfs.TestReplication Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767377/HDFS-9263-001.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 8175c4f Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13059/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13059/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/13059/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13059/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13059/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13059/console This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Hi Steve Loughran.

          I'm a little unclear on what this patch is trying to solve. The various test working dir properties are all built up from a base of project.build.directory in hadoop-project/pom.xml. I'd expect that to avoid dirtying the workspace by writing files outside of target. It's true that there are too many redundant properties like Andrew pointed out, but I don't think consolidating those was the original intent of this issue. Is there a specific test that you see writing outside of target?

          Or is the issue specifically that this causes a problem only when running outside the Maven lifecycle?

          many buildups of test dirs now use something random, rather than a hard-coded path like "dfs". This includes minidfs cluster...which should improve parallelism on test runs.

          I don't think we'll be able to do this. As you've seen, there are various tests that need to do a NameNode restart without reformatting so that we can verify things like correct metadata reloaded from fsimage on startup. If this is something needed to support runs outside of Maven, then would it be possible to statically intialize a random directory once at startup and then reuse it for the whole suite? Maybe that would help.

          When running through Maven, the build will generate N different unique directories, where N = number of concurrent test processes. Then, each concurrent test process really operates in its own unique testing directory. That has been sufficient for isolation in the Maven runs. (Again, maybe you're trying to improve support for running outside of Maven?)

          Show
          cnauroth Chris Nauroth added a comment - Hi Steve Loughran . I'm a little unclear on what this patch is trying to solve. The various test working dir properties are all built up from a base of project.build.directory in hadoop-project/pom.xml. I'd expect that to avoid dirtying the workspace by writing files outside of target . It's true that there are too many redundant properties like Andrew pointed out, but I don't think consolidating those was the original intent of this issue. Is there a specific test that you see writing outside of target ? Or is the issue specifically that this causes a problem only when running outside the Maven lifecycle? many buildups of test dirs now use something random, rather than a hard-coded path like "dfs". This includes minidfs cluster...which should improve parallelism on test runs. I don't think we'll be able to do this. As you've seen, there are various tests that need to do a NameNode restart without reformatting so that we can verify things like correct metadata reloaded from fsimage on startup. If this is something needed to support runs outside of Maven, then would it be possible to statically intialize a random directory once at startup and then reuse it for the whole suite? Maybe that would help. When running through Maven, the build will generate N different unique directories, where N = number of concurrent test processes. Then, each concurrent test process really operates in its own unique testing directory. That has been sufficient for isolation in the Maven runs. (Again, maybe you're trying to improve support for running outside of Maven?)
          Hide
          cnauroth Chris Nauroth added a comment -

          I just noticed the Jenkins run on HADOOP-11880 shows the release audit warning problems. I guess we'd have to backtrace from those files to figure out which tests are naughty. (If you've already done so, please let me know.)

          Show
          cnauroth Chris Nauroth added a comment - I just noticed the Jenkins run on HADOOP-11880 shows the release audit warning problems. I guess we'd have to backtrace from those files to figure out which tests are naughty. (If you've already done so, please let me know.)
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Chris, the test/build/data path appeared throughout the code as the default value if the sysprop wasn't set, it also appeared elsewhere. Consolidating it means that there is one place for the default. As you note, this shouldn't be the cause of the leftovers. Doing a bulk search for the path means that we are guaranteed to have found the uses; consolidation means it won't ever arise again.

          Going through that code shows that a fair few of the tests have hard-coded expectations of the path set up by minidfs cluster, specifically $

          {test.build.data}

          /dfs. That's going to be an intermittent breaker of parallel test runs, as minidfs clusters may come up on separate ports, but they could be stamping on each other's datasets. We hadn't really spotted that to date (and the audits wont show it), but it's there.

          If we not only consolidate test dir setup, but do it in a way that isolates it for each test suite, we get that isolation. Trouble is, the -001 patch does give isolation for every MiniDFS cluster, but breaks those tests which expect restarted DFS clusters to have the same file path. We're going to need to come up with the unique path for each test suite or method, not per cluster instance —or at least add an option for that.

          Show
          stevel@apache.org Steve Loughran added a comment - Chris, the test/build/data path appeared throughout the code as the default value if the sysprop wasn't set, it also appeared elsewhere. Consolidating it means that there is one place for the default. As you note, this shouldn't be the cause of the leftovers. Doing a bulk search for the path means that we are guaranteed to have found the uses; consolidation means it won't ever arise again. Going through that code shows that a fair few of the tests have hard-coded expectations of the path set up by minidfs cluster, specifically $ {test.build.data} /dfs. That's going to be an intermittent breaker of parallel test runs, as minidfs clusters may come up on separate ports, but they could be stamping on each other's datasets. We hadn't really spotted that to date (and the audits wont show it), but it's there. If we not only consolidate test dir setup, but do it in a way that isolates it for each test suite, we get that isolation. Trouble is, the -001 patch does give isolation for every MiniDFS cluster, but breaks those tests which expect restarted DFS clusters to have the same file path. We're going to need to come up with the unique path for each test suite or method, not per cluster instance —or at least add an option for that.
          Hide
          cnauroth Chris Nauroth added a comment -

          Steve Loughran, regarding the side discussion on HADOOP-11880, I have traced the problem to TestMiniDFSCluster, and the problem only occurs while running with the HDFS-9263 patch applied. I hope you don't mind, but I'm attaching a v002 patch with a small modification to fix it.

          My only change is in GenericTestUtils. Prepare to smack forehead. Here is the patch v001 code:

            public static final String DEFAULT_TEST_DATA_DIR =
                "target " + File.pathSeparator + "test" + File.pathSeparator + "data";
          

          Here is my change in v002:

            public static final String DEFAULT_TEST_DATA_DIR =
                "target" + File.separator + "test" + File.separator + "data";
          

          I removed the extra space character at the end of the "target" string literal, and I switched from File.pathSeparator (i.e. classpath separator, ':' on *nixes) to File.separator (i.e.file system path separator, '/' on *nixes). I constantly mix up those 2 myself. I wish they had clearer names.

          As to why TestMiniDFSCluster exposed this, one of the tests in that suite specifically removes the test.build.data property to check if the mini-cluster can still start using defaults. After running that test suite, I could see it created the funny paths containing spaces and colons. For code that passes the path through a URI, it would end up encoding the space to %20 too.

          If we not only consolidate test dir setup, but do it in a way that isolates it for each test suite, we get that isolation.

          I'm on board with the consolidation aspect, but it's still unclear to me that there is a benefit of adding another random string into the path. I suppose if the sub-directory was named to match the test suite, then that has some benefit for post-mortem analysis after a test failure. You could go back and inspect metadata and blocks, and you'd know that you were looking at files specific to that test suite.

          OTOH, this has the side effect of using many more directories, and they won't get cleaned up in between runs of different suites. Typically, the data gets wiped between suite runs, either explicitly via FileUtil#fullyDelete, or implicitly via things like a NameNode format. I tried a full test run of hadoop-hdfs, and then I saw this:

          > du -hs ~/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/target/test/data
          6.1G	/home/cnauroth/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/target/test/data
          

          That's more disk consumption than I'm used to seeing from a test run. I'm pretty sure I'd need to reallocate volumes on some of my wimpier VMs to accommodate this.

          Show
          cnauroth Chris Nauroth added a comment - Steve Loughran , regarding the side discussion on HADOOP-11880 , I have traced the problem to TestMiniDFSCluster , and the problem only occurs while running with the HDFS-9263 patch applied. I hope you don't mind, but I'm attaching a v002 patch with a small modification to fix it. My only change is in GenericTestUtils . Prepare to smack forehead. Here is the patch v001 code: public static final String DEFAULT_TEST_DATA_DIR = "target " + File.pathSeparator + "test" + File.pathSeparator + "data" ; Here is my change in v002: public static final String DEFAULT_TEST_DATA_DIR = "target" + File.separator + "test" + File.separator + "data" ; I removed the extra space character at the end of the "target" string literal, and I switched from File.pathSeparator (i.e. classpath separator, ':' on *nixes) to File.separator (i.e.file system path separator, '/' on *nixes). I constantly mix up those 2 myself. I wish they had clearer names. As to why TestMiniDFSCluster exposed this, one of the tests in that suite specifically removes the test.build.data property to check if the mini-cluster can still start using defaults. After running that test suite, I could see it created the funny paths containing spaces and colons. For code that passes the path through a URI , it would end up encoding the space to %20 too. If we not only consolidate test dir setup, but do it in a way that isolates it for each test suite, we get that isolation. I'm on board with the consolidation aspect, but it's still unclear to me that there is a benefit of adding another random string into the path. I suppose if the sub-directory was named to match the test suite, then that has some benefit for post-mortem analysis after a test failure. You could go back and inspect metadata and blocks, and you'd know that you were looking at files specific to that test suite. OTOH, this has the side effect of using many more directories, and they won't get cleaned up in between runs of different suites. Typically, the data gets wiped between suite runs, either explicitly via FileUtil#fullyDelete , or implicitly via things like a NameNode format. I tried a full test run of hadoop-hdfs, and then I saw this: > du -hs ~/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/target/test/data 6.1G /home/cnauroth/git/hadoop-common/hadoop-hdfs-project/hadoop-hdfs/target/test/data That's more disk consumption than I'm used to seeing from a test run. I'm pretty sure I'd need to reallocate volumes on some of my wimpier VMs to accommodate this.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 10m 50s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 36 new or modified test files.
          +1 javac 8m 58s There were no new javac warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 2m 45s There were no new checkstyle issues.
          -1 whitespace 0m 6s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 47s mvn install still works.
          +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
          +1 findbugs 5m 9s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 8m 13s Tests failed in hadoop-common.
          -1 hdfs tests 45m 57s Tests failed in hadoop-hdfs.
              84m 53s  



          Reason Tests
          Failed unit tests hadoop.fs.TestLocalFsFCStatistics
            hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            hadoop.hdfs.TestDFSRename
            hadoop.hdfs.TestDatanodeConfig
            hadoop.hdfs.server.namenode.TestXAttrConfigFlag
            hadoop.hdfs.TestBlockReaderFactory
            hadoop.hdfs.TestDFSClientRetries
            hadoop.hdfs.server.namenode.TestFSImage
            hadoop.hdfs.TestFileCreationDelete
            hadoop.hdfs.TestSnapshotCommands
            hadoop.hdfs.TestFileConcurrentReader
            hadoop.hdfs.server.namenode.TestCheckpoint
            hadoop.hdfs.TestFileAppend2
            hadoop.hdfs.tools.TestDFSAdmin
            hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.TestLeaseRecovery
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.tools.TestDelegationTokenFetcher
            hadoop.tracing.TestTraceAdmin
            hadoop.hdfs.TestWriteConfigurationToDFS
            hadoop.hdfs.tools.TestStoragePolicyCommands
            hadoop.hdfs.TestReplication
            hadoop.hdfs.server.namenode.TestAclConfigFlag
            hadoop.hdfs.TestPipelines
            hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.tools.TestGetGroups
            hadoop.hdfs.TestParallelShortCircuitRead
            hadoop.hdfs.TestRenameWhileOpen
            hadoop.hdfs.TestBlocksScheduledCounter
            hadoop.hdfs.TestRollingUpgradeRollback
            hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer
            hadoop.hdfs.TestErasureCodingPolicies
            hadoop.hdfs.tools.TestDFSAdminWithHA
            hadoop.hdfs.util.TestBestEffortLongFile
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.TestDataTransferKeepalive
            hadoop.hdfs.TestFSInputChecker
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.fs.permission.TestStickyBit
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
            hadoop.hdfs.TestRead
            hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.server.namenode.TestFSEditLogLoader
            hadoop.hdfs.TestExternalBlockReader
            hadoop.hdfs.TestDatanodeRegistration
            hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.server.namenode.TestFileContextAcl
            hadoop.hdfs.tools.TestDFSHAAdminMiniCluster
            hadoop.hdfs.TestSmallBlock
            hadoop.hdfs.TestDFSRollback
            hadoop.hdfs.TestDFSStorageStateRecovery
            hadoop.hdfs.TestHdfsAdmin
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
            hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary
            hadoop.hdfs.TestConnCache
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks
            hadoop.hdfs.TestRecoverStripedFile
            hadoop.hdfs.TestRollingUpgradeDowngrade
            hadoop.hdfs.TestDFSFinalize
            hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
            hadoop.hdfs.server.datanode.TestCachingStrategy
            hadoop.hdfs.TestBlockMissingException
            hadoop.hdfs.TestFileAppend3
            hadoop.hdfs.TestFileLengthOnClusterRestart
            hadoop.hdfs.server.namenode.TestNameEditsConfigs
            hadoop.hdfs.web.TestWebHDFSXAttr
            hadoop.hdfs.crypto.TestHdfsCryptoStreams
            hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs
            hadoop.hdfs.web.TestWebHDFSAcl
            hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.tools.TestDebugAdmin
            hadoop.hdfs.server.namenode.TestStartup
            hadoop.hdfs.TestDFSStripedOutputStream
            hadoop.hdfs.qjournal.TestMiniJournalCluster
            hadoop.hdfs.TestPersistBlocks
            hadoop.hdfs.TestInjectionForSimulatedStorage
            hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
            hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
            hadoop.hdfs.TestDFSPermission
            hadoop.hdfs.TestFileCreation
            hadoop.hdfs.TestDFSInotifyEventInputStream
            hadoop.hdfs.server.namenode.TestEditLogJournalFailures
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.TestDFSClientExcludedNodes
            hadoop.hdfs.TestSafeModeWithStripedFile
            hadoop.hdfs.TestDFSStripedInputStream
            hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
            hadoop.hdfs.TestMiniDFSCluster
            hadoop.hdfs.TestParallelRead
            hadoop.hdfs.qjournal.server.TestJournalNodeMXBean
            hadoop.hdfs.TestLeaseRecovery2
            hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
            hadoop.hdfs.TestRestartDFS
            hadoop.tracing.TestTracing
            hadoop.hdfs.TestReservedRawPaths
            hadoop.hdfs.server.namenode.snapshot.TestSnapshot
            hadoop.hdfs.TestEncryptionZonesWithHA
            hadoop.hdfs.TestParallelShortCircuitReadNoChecksum
            hadoop.hdfs.TestHFlush
            hadoop.hdfs.TestParallelUnixDomainRead
            hadoop.hdfs.TestDatanodeReport
            hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
            hadoop.hdfs.TestDatanodeLayoutUpgrade
            hadoop.hdfs.TestFileAppendRestart
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000
            hadoop.hdfs.TestRemoteBlockReader2
            hadoop.hdfs.TestSetrepIncreasing
            hadoop.hdfs.security.TestDelegationToken
            hadoop.hdfs.server.namenode.TestParallelImageWrite
            hadoop.hdfs.shortcircuit.TestShortCircuitCache
            hadoop.hdfs.TestReadWhileWriting
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForXAttr
            hadoop.hdfs.TestLease
            hadoop.hdfs.server.namenode.TestNameNodeAcl
            hadoop.hdfs.TestEncryptedTransfer
            hadoop.hdfs.TestWriteReadStripedFile
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.TestEncryptionZonesWithKMS
            hadoop.hdfs.TestClientBlockVerification
            hadoop.hdfs.TestGetBlocks
            hadoop.hdfs.server.namenode.TestFileContextXAttr
            hadoop.hdfs.TestMissingBlocksAlert
            hadoop.hdfs.security.token.block.TestBlockToken
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.tools.TestDFSZKFailoverController
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.TestDFSShell
            hadoop.hdfs.server.namenode.TestMetadataVersionOutput
            hadoop.hdfs.TestBlockReaderLocal
            hadoop.hdfs.server.namenode.TestNameNodeXAttr
            hadoop.hdfs.TestHDFSFileSystemContract
            hadoop.hdfs.TestDFSStartupVersions
            hadoop.hdfs.server.blockmanagement.TestNodeCount
            hadoop.hdfs.TestAppendSnapshotTruncate
            hadoop.fs.TestHDFSFileContextMainOperations
            hadoop.hdfs.TestDatanodeDeath
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
            hadoop.hdfs.server.namenode.TestBackupNode
            hadoop.hdfs.TestSafeMode
            hadoop.hdfs.TestSeekBug
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl
            hadoop.hdfs.TestDisableConnCache
            hadoop.hdfs.TestLargeBlock
            hadoop.hdfs.TestMultiThreadedHflush
            hadoop.tracing.TestTracingShortCircuitLocalRead
            hadoop.hdfs.TestClose
            hadoop.hdfs.TestSetTimes
            hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
          Timed out tests org.apache.hadoop.hdfs.server.namenode.TestFsck



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768142/HDFS-9263-002.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 513ec3d
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13138/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13138/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/13138/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13138/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13138/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13138/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 10m 50s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 36 new or modified test files. +1 javac 8m 58s There were no new javac warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 45s There were no new checkstyle issues. -1 whitespace 0m 6s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 47s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 5m 9s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 8m 13s Tests failed in hadoop-common. -1 hdfs tests 45m 57s Tests failed in hadoop-hdfs.     84m 53s   Reason Tests Failed unit tests hadoop.fs.TestLocalFsFCStatistics   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   hadoop.hdfs.TestDFSRename   hadoop.hdfs.TestDatanodeConfig   hadoop.hdfs.server.namenode.TestXAttrConfigFlag   hadoop.hdfs.TestBlockReaderFactory   hadoop.hdfs.TestDFSClientRetries   hadoop.hdfs.server.namenode.TestFSImage   hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.TestSnapshotCommands   hadoop.hdfs.TestFileConcurrentReader   hadoop.hdfs.server.namenode.TestCheckpoint   hadoop.hdfs.TestFileAppend2   hadoop.hdfs.tools.TestDFSAdmin   hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestLeaseRecovery   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.tools.TestDelegationTokenFetcher   hadoop.tracing.TestTraceAdmin   hadoop.hdfs.TestWriteConfigurationToDFS   hadoop.hdfs.tools.TestStoragePolicyCommands   hadoop.hdfs.TestReplication   hadoop.hdfs.server.namenode.TestAclConfigFlag   hadoop.hdfs.TestPipelines   hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.tools.TestGetGroups   hadoop.hdfs.TestParallelShortCircuitRead   hadoop.hdfs.TestRenameWhileOpen   hadoop.hdfs.TestBlocksScheduledCounter   hadoop.hdfs.TestRollingUpgradeRollback   hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer   hadoop.hdfs.TestErasureCodingPolicies   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.util.TestBestEffortLongFile   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.TestDataTransferKeepalive   hadoop.hdfs.TestFSInputChecker   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.fs.permission.TestStickyBit   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead   hadoop.hdfs.TestRead   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.namenode.TestFSEditLogLoader   hadoop.hdfs.TestExternalBlockReader   hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.server.namenode.TestFileContextAcl   hadoop.hdfs.tools.TestDFSHAAdminMiniCluster   hadoop.hdfs.TestSmallBlock   hadoop.hdfs.TestDFSRollback   hadoop.hdfs.TestDFSStorageStateRecovery   hadoop.hdfs.TestHdfsAdmin   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer   hadoop.hdfs.server.namenode.TestFSImageWithSnapshot   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary   hadoop.hdfs.TestConnCache   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks   hadoop.hdfs.TestRecoverStripedFile   hadoop.hdfs.TestRollingUpgradeDowngrade   hadoop.hdfs.TestDFSFinalize   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   hadoop.hdfs.server.datanode.TestCachingStrategy   hadoop.hdfs.TestBlockMissingException   hadoop.hdfs.TestFileAppend3   hadoop.hdfs.TestFileLengthOnClusterRestart   hadoop.hdfs.server.namenode.TestNameEditsConfigs   hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.crypto.TestHdfsCryptoStreams   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs   hadoop.hdfs.web.TestWebHDFSAcl   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.tools.TestDebugAdmin   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.TestDFSStripedOutputStream   hadoop.hdfs.qjournal.TestMiniJournalCluster   hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.TestInjectionForSimulatedStorage   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer   hadoop.hdfs.TestDFSPermission   hadoop.hdfs.TestFileCreation   hadoop.hdfs.TestDFSInotifyEventInputStream   hadoop.hdfs.server.namenode.TestEditLogJournalFailures   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestDFSClientExcludedNodes   hadoop.hdfs.TestSafeModeWithStripedFile   hadoop.hdfs.TestDFSStripedInputStream   hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot   hadoop.hdfs.TestMiniDFSCluster   hadoop.hdfs.TestParallelRead   hadoop.hdfs.qjournal.server.TestJournalNodeMXBean   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot   hadoop.hdfs.TestRestartDFS   hadoop.tracing.TestTracing   hadoop.hdfs.TestReservedRawPaths   hadoop.hdfs.server.namenode.snapshot.TestSnapshot   hadoop.hdfs.TestEncryptionZonesWithHA   hadoop.hdfs.TestParallelShortCircuitReadNoChecksum   hadoop.hdfs.TestHFlush   hadoop.hdfs.TestParallelUnixDomainRead   hadoop.hdfs.TestDatanodeReport   hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens   hadoop.hdfs.TestDatanodeLayoutUpgrade   hadoop.hdfs.TestFileAppendRestart   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000   hadoop.hdfs.TestRemoteBlockReader2   hadoop.hdfs.TestSetrepIncreasing   hadoop.hdfs.security.TestDelegationToken   hadoop.hdfs.server.namenode.TestParallelImageWrite   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.TestReadWhileWriting   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForXAttr   hadoop.hdfs.TestLease   hadoop.hdfs.server.namenode.TestNameNodeAcl   hadoop.hdfs.TestEncryptedTransfer   hadoop.hdfs.TestWriteReadStripedFile   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.hdfs.TestClientBlockVerification   hadoop.hdfs.TestGetBlocks   hadoop.hdfs.server.namenode.TestFileContextXAttr   hadoop.hdfs.TestMissingBlocksAlert   hadoop.hdfs.security.token.block.TestBlockToken   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.tools.TestDFSZKFailoverController   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.TestDFSShell   hadoop.hdfs.server.namenode.TestMetadataVersionOutput   hadoop.hdfs.TestBlockReaderLocal   hadoop.hdfs.server.namenode.TestNameNodeXAttr   hadoop.hdfs.TestHDFSFileSystemContract   hadoop.hdfs.TestDFSStartupVersions   hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.TestAppendSnapshotTruncate   hadoop.fs.TestHDFSFileContextMainOperations   hadoop.hdfs.TestDatanodeDeath   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.hdfs.TestSafeMode   hadoop.hdfs.TestSeekBug   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl   hadoop.hdfs.TestDisableConnCache   hadoop.hdfs.TestLargeBlock   hadoop.hdfs.TestMultiThreadedHflush   hadoop.tracing.TestTracingShortCircuitLocalRead   hadoop.hdfs.TestClose   hadoop.hdfs.TestSetTimes   hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks Timed out tests org.apache.hadoop.hdfs.server.namenode.TestFsck Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768142/HDFS-9263-002.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 513ec3d Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13138/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13138/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/13138/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13138/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13138/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13138/console This message was automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          We should backport this to maintenance releases too to keep the test-runs in sync.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - We should backport this to maintenance releases too to keep the test-runs in sync.
          Hide
          sjlee0 Sangjin Lee added a comment -

          Targeting 2.6.3 now that 2.6.2 has shipped.

          Show
          sjlee0 Sangjin Lee added a comment - Targeting 2.6.3 now that 2.6.2 has shipped.
          Hide
          aw Allen Wittenauer added a comment -

          Could we at least get a fix in that moves the directory so that people stop asking the Yetus team to "fix" this?

          Show
          aw Allen Wittenauer added a comment - Could we at least get a fix in that moves the directory so that people stop asking the Yetus team to "fix" this?
          Hide
          cnauroth Chris Nauroth added a comment -

          Steve Loughran, what do you think about going ahead with Allen's suggestion of a minimal patch just to make sure we only write under target/? I'm concerned about the increased space consumption mentioned in my last comment, but a bug fix to stay under target/ would be uncontroversial. If you'd like me to help out with a patch, let me know.

          Show
          cnauroth Chris Nauroth added a comment - Steve Loughran , what do you think about going ahead with Allen's suggestion of a minimal patch just to make sure we only write under target/? I'm concerned about the increased space consumption mentioned in my last comment, but a bug fix to stay under target/ would be uncontroversial. If you'd like me to help out with a patch, let me know.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          OK. I think one that didn't go near minihdfs would avoid a lot of the test failures, as it would be consistently placed

          Show
          stevel@apache.org Steve Loughran added a comment - OK. I think one that didn't go near minihdfs would avoid a lot of the test failures, as it would be consistently placed
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user steveloughran opened a pull request:

          https://github.com/apache/hadoop/pull/53

          HDFS-9263

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/steveloughran/hadoop jenkins/HDFS-9263_build_test_data

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/hadoop/pull/53.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #53


          commit a2a7df4f521602fef3bd49c708b0fecb080c50a1
          Author: Steve Loughran <stevel@hortonworks.com>
          Date: 2015-10-19T14:06:08Z

          HDFS-9263: set up GTU for single place for test dir/path setup

          commit c2391515fc2eed5fa1249020585cccd94a6b523d
          Author: Steve Loughran <stevel@hortonworks.com>
          Date: 2015-10-19T14:07:19Z

          HDFS-9263 factoring out refs to build test propertyes & paths in hadoop-common

          commit 646811a57765532c6b2be701b67a964435b1ed50
          Author: Steve Loughran <stevel@hortonworks.com>
          Date: 2015-10-19T14:07:37Z

          HDFS-9263 hdft-tests

          commit cac0e86d6fa51ae97df9995ddea3f0df3ad6601d
          Author: Steve Loughran <stevel@hortonworks.com>
          Date: 2015-10-19T19:03:35Z

          HDFS-9263 mapreduce tests

          commit ce93191dbf3e6d631736561d795573369286de88
          Author: Steve Loughran <stevel@hortonworks.com>
          Date: 2015-10-19T19:03:53Z

          HDFS-9263 hadoop archive test

          commit 6eb8f0ecb9e220779740e263cdcd68b23ef9e681
          Author: Steve Loughran <stevel@hortonworks.com>
          Date: 2015-11-22T18:34:40Z

          HDFS-9263 MiniDFS cluster to use hardcoded subdir under test dir.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user steveloughran opened a pull request: https://github.com/apache/hadoop/pull/53 HDFS-9263 You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveloughran/hadoop jenkins/ HDFS-9263 _build_test_data Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/53.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #53 commit a2a7df4f521602fef3bd49c708b0fecb080c50a1 Author: Steve Loughran <stevel@hortonworks.com> Date: 2015-10-19T14:06:08Z HDFS-9263 : set up GTU for single place for test dir/path setup commit c2391515fc2eed5fa1249020585cccd94a6b523d Author: Steve Loughran <stevel@hortonworks.com> Date: 2015-10-19T14:07:19Z HDFS-9263 factoring out refs to build test propertyes & paths in hadoop-common commit 646811a57765532c6b2be701b67a964435b1ed50 Author: Steve Loughran <stevel@hortonworks.com> Date: 2015-10-19T14:07:37Z HDFS-9263 hdft-tests commit cac0e86d6fa51ae97df9995ddea3f0df3ad6601d Author: Steve Loughran <stevel@hortonworks.com> Date: 2015-10-19T19:03:35Z HDFS-9263 mapreduce tests commit ce93191dbf3e6d631736561d795573369286de88 Author: Steve Loughran <stevel@hortonworks.com> Date: 2015-10-19T19:03:53Z HDFS-9263 hadoop archive test commit 6eb8f0ecb9e220779740e263cdcd68b23ef9e681 Author: Steve Loughran <stevel@hortonworks.com> Date: 2015-11-22T18:34:40Z HDFS-9263 MiniDFS cluster to use hardcoded subdir under test dir.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user cnauroth commented on a diff in the pull request:

          https://github.com/apache/hadoop/pull/53#discussion_r45787996

          — Diff: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java —
          @@ -59,6 +59,22 @@

          private static final AtomicInteger sequence = new AtomicInteger();

          + /**
          + * system property for test data:

          {@value}
          + */
          + public static final String SYSPROP_TEST_DATA_DIR = "test.build.data";
          +
          + /**
          + * Default path for test data: {@value}

          + */
          + public static final String DEFAULT_TEST_DATA_DIR =
          + "target " + File.pathSeparator + "test" + File.pathSeparator + "data";
          — End diff –

          I think you'll still need to incorporate the fixes I suggested earlier in a JIRA comment: remove the extra space character at the end of the `"target "` string literal, and switch from `File.pathSeparator` to `File.separator`.

          Show
          githubbot ASF GitHub Bot added a comment - Github user cnauroth commented on a diff in the pull request: https://github.com/apache/hadoop/pull/53#discussion_r45787996 — Diff: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java — @@ -59,6 +59,22 @@ private static final AtomicInteger sequence = new AtomicInteger(); + /** + * system property for test data: {@value} + */ + public static final String SYSPROP_TEST_DATA_DIR = "test.build.data"; + + /** + * Default path for test data: {@value} + */ + public static final String DEFAULT_TEST_DATA_DIR = + "target " + File.pathSeparator + "test" + File.pathSeparator + "data"; — End diff – I think you'll still need to incorporate the fixes I suggested earlier in a JIRA comment: remove the extra space character at the end of the `"target "` string literal, and switch from `File.pathSeparator` to `File.separator`.
          Hide
          djp Junping Du added a comment -

          Hi, can we move this out of 2.6.3? Thanks!

          Show
          djp Junping Du added a comment - Hi, can we move this out of 2.6.3? Thanks!
          Hide
          djp Junping Du added a comment -

          Move it out to 2.6.4 given the JIRA is not updated for a period of time.

          Show
          djp Junping Du added a comment - Move it out to 2.6.4 given the JIRA is not updated for a period of time.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          ..just applied chris's 002 patch to trunk and pushed out on same PR branch

          Show
          stevel@apache.org Steve Loughran added a comment - ..just applied chris's 002 patch to trunk and pushed out on same PR branch
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Chris, the latest patch I've pushed out onto the PR

          1. has all your changes in
          2. applies to trunk
          3. reverts the minidfs cluster random dir. This means that there's on a very few tests asking for random paths now.
          Show
          stevel@apache.org Steve Loughran added a comment - Chris, the latest patch I've pushed out onto the PR has all your changes in applies to trunk reverts the minidfs cluster random dir. This means that there's on a very few tests asking for random paths now.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 6s HDFS-9263 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HDFS-9263
          GITHUB PR https://github.com/apache/hadoop/pull/53
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14333/console
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14333/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 6s HDFS-9263 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-9263 GITHUB PR https://github.com/apache/hadoop/pull/53 Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14333/console Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14333/console This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Hi Steve Loughran, how is the issue going?

          Show
          ajisakaa Akira Ajisaka added a comment - Hi Steve Loughran , how is the issue going?
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Not looked at this for a while; been busy with non-hadoop stuff. You want to take over?

          Show
          stevel@apache.org Steve Loughran added a comment - Not looked at this for a while; been busy with non-hadoop stuff. You want to take over?
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Either ways, can you please see if this can get into 2.7.3 within a week? Tx.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Either ways, can you please see if this can get into 2.7.3 within a week? Tx.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Hi Steve Loughran,
          As of current jenkins reports seen, there is no such files created outside "target" directory which could not be cleaned by "mvn clean".
          If such files present jenkins would have caught it under "ASF LICENCE warnings".
          Still you want this to be fixed before 2.7.3 ?

          Patch does a nice a Job of centralizing the usage of system property "test.build.data".
          But seeing the intention of this Jira, and current state of jenkins reports, I feel think this might not be a blocker for 2.7.3.

          Show
          vinayrpet Vinayakumar B added a comment - Hi Steve Loughran , As of current jenkins reports seen, there is no such files created outside "target" directory which could not be cleaned by "mvn clean". If such files present jenkins would have caught it under "ASF LICENCE warnings". Still you want this to be fixed before 2.7.3 ? Patch does a nice a Job of centralizing the usage of system property "test.build.data". But seeing the intention of this Jira, and current state of jenkins reports, I feel think this might not be a blocker for 2.7.3.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Attaching the Updated patch.
          Replaced all occurances of "test.build.data" with GenericTestUtils

          Show
          vinayrpet Vinayakumar B added a comment - Attaching the Updated patch. Replaced all occurances of "test.build.data" with GenericTestUtils
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 6s HDFS-9263 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HDFS-9263
          GITHUB PR https://github.com/apache/hadoop/pull/53
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15016/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 6s HDFS-9263 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-9263 GITHUB PR https://github.com/apache/hadoop/pull/53 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15016/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 6s HDFS-9263 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HDFS-9263
          GITHUB PR https://github.com/apache/hadoop/pull/53
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15017/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 6s HDFS-9263 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-9263 GITHUB PR https://github.com/apache/hadoop/pull/53 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15017/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 5s HDFS-9263 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HDFS-9263
          GITHUB PR https://github.com/apache/hadoop/pull/53
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15018/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s HDFS-9263 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-9263 GITHUB PR https://github.com/apache/hadoop/pull/53 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15018/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Vinayakumar for taking this over.
          Looks like Jenkins job is done against pull request even when a patch is attached.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Vinayakumar for taking this over. Looks like Jenkins job is done against pull request even when a patch is attached.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Looks like Jenkins job is done against pull request even when a patch is attached.

          Yes. I will check it.

          Show
          vinayrpet Vinayakumar B added a comment - Looks like Jenkins job is done against pull request even when a patch is attached. Yes. I will check it.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Filed YETUS-362.

          Show
          ajisakaa Akira Ajisaka added a comment - Filed YETUS-362 .
          Hide
          ajisakaa Akira Ajisaka added a comment -

          But seeing the intention of this Jira, and current state of jenkins reports, I feel think this might not be a blocker for 2.7.3.

          Agree. I think we can reduce the priority.

          Show
          ajisakaa Akira Ajisaka added a comment - But seeing the intention of this Jira, and current state of jenkins reports, I feel think this might not be a blocker for 2.7.3. Agree. I think we can reduce the priority.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Given the purpose of this issue has changed, I moved this jira from HDFS to Common and updated the title and description.

          Show
          ajisakaa Akira Ajisaka added a comment - Given the purpose of this issue has changed, I moved this jira from HDFS to Common and updated the title and description.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 7s HADOOP-12984 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HADOOP-12984
          GITHUB PR https://github.com/apache/hadoop/pull/53
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8989/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 7s HADOOP-12984 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HADOOP-12984 GITHUB PR https://github.com/apache/hadoop/pull/53 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8989/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Renamed the patch to avoid downloading patch from the pull request.

          Show
          ajisakaa Akira Ajisaka added a comment - Renamed the patch to avoid downloading patch from the pull request.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 6s HADOOP-12984 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HADOOP-12984
          GITHUB PR https://github.com/apache/hadoop/pull/53
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8996/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 6s HADOOP-12984 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HADOOP-12984 GITHUB PR https://github.com/apache/hadoop/pull/53 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8996/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user vinayakumarb opened a pull request:

          https://github.com/apache/hadoop/pull/88

          HADOOP-12984

          Add GenericTestUtils.getTestDir method and use it for temporary directory in tests

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/vinayakumarb/hadoop trunk

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/hadoop/pull/88.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #88


          commit ef71c4bdff9b0e5a40322d6e210349d4907b6bfb
          Author: Vinayakumar B <vinayakumarb@apache.org>
          Date: 2016-04-01T05:22:46Z

          HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for temporary directory in tests


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user vinayakumarb opened a pull request: https://github.com/apache/hadoop/pull/88 HADOOP-12984 Add GenericTestUtils.getTestDir method and use it for temporary directory in tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/vinayakumarb/hadoop trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/88.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #88 commit ef71c4bdff9b0e5a40322d6e210349d4907b6bfb Author: Vinayakumar B <vinayakumarb@apache.org> Date: 2016-04-01T05:22:46Z HADOOP-12984 . Add GenericTestUtils.getTestDir method and use it for temporary directory in tests
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 97 new or modified test files.
          0 mvndep 0m 30s Maven dependency ordering for branch
          +1 mvninstall 7m 1s trunk passed
          +1 compile 6m 8s trunk passed with JDK v1.8.0_77
          +1 compile 6m 40s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 10s trunk passed
          +1 mvnsite 2m 34s trunk passed
          +1 mvneclipse 0m 40s trunk passed
          +1 findbugs 5m 29s trunk passed
          +1 javadoc 2m 25s trunk passed with JDK v1.8.0_77
          +1 javadoc 3m 29s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 2m 13s the patch passed
          +1 compile 6m 49s the patch passed with JDK v1.8.0_77
          +1 javac 9m 17s root-jdk1.8.0_77 with JDK v1.8.0_77 generated 0 new + 738 unchanged - 1 fixed = 738 total (was 739)
          +1 javac 6m 49s root in the patch passed with JDK v1.8.0_77.
          +1 compile 7m 18s the patch passed with JDK v1.7.0_95
          +1 javac 16m 35s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735)
          +1 javac 7m 18s root in the patch passed with JDK v1.7.0_95.
          +1 checkstyle 1m 4s the patch passed
          +1 mvnsite 2m 33s the patch passed
          +1 mvneclipse 0m 42s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 6m 25s the patch passed
          +1 javadoc 2m 36s the patch passed with JDK v1.8.0_77
          +1 javadoc 3m 28s the patch passed with JDK v1.7.0_95
          +1 unit 8m 38s hadoop-common in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 53s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77.
          -1 unit 60m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_77.
          -1 unit 8m 0s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 51m 9s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 25s Patch does not generate ASF License warnings.
          202m 12s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.TestDatanodeLayoutUpgrade
            hadoop.hdfs.qjournal.TestMiniJournalCluster
            hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.TestBackupNode
            hadoop.TestRefreshCallQueue
            hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.server.namenode.TestStartup
            hadoop.hdfs.shortcircuit.TestShortCircuitCache
            hadoop.hdfs.server.namenode.snapshot.TestSnapshot
          JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.TestDatanodeLayoutUpgrade
            hadoop.hdfs.TestHFlush
            hadoop.hdfs.qjournal.TestMiniJournalCluster
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.TestBackupNode
            hadoop.TestRefreshCallQueue
            hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.server.namenode.TestStartup
            hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            hadoop.hdfs.server.namenode.snapshot.TestSnapshot



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Issue HADOOP-12984
          GITHUB PR https://github.com/apache/hadoop/pull/88
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 520e89b3f77f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a8d8b80
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 97 new or modified test files. 0 mvndep 0m 30s Maven dependency ordering for branch +1 mvninstall 7m 1s trunk passed +1 compile 6m 8s trunk passed with JDK v1.8.0_77 +1 compile 6m 40s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 10s trunk passed +1 mvnsite 2m 34s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 5m 29s trunk passed +1 javadoc 2m 25s trunk passed with JDK v1.8.0_77 +1 javadoc 3m 29s trunk passed with JDK v1.7.0_95 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 2m 13s the patch passed +1 compile 6m 49s the patch passed with JDK v1.8.0_77 +1 javac 9m 17s root-jdk1.8.0_77 with JDK v1.8.0_77 generated 0 new + 738 unchanged - 1 fixed = 738 total (was 739) +1 javac 6m 49s root in the patch passed with JDK v1.8.0_77. +1 compile 7m 18s the patch passed with JDK v1.7.0_95 +1 javac 16m 35s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735) +1 javac 7m 18s root in the patch passed with JDK v1.7.0_95. +1 checkstyle 1m 4s the patch passed +1 mvnsite 2m 33s the patch passed +1 mvneclipse 0m 42s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 6m 25s the patch passed +1 javadoc 2m 36s the patch passed with JDK v1.8.0_77 +1 javadoc 3m 28s the patch passed with JDK v1.7.0_95 +1 unit 8m 38s hadoop-common in the patch passed with JDK v1.8.0_77. +1 unit 0m 53s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. -1 unit 60m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_77. -1 unit 8m 0s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 51m 9s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 202m 12s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.TestDatanodeLayoutUpgrade   hadoop.hdfs.qjournal.TestMiniJournalCluster   hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.TestRefreshCallQueue   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.server.namenode.snapshot.TestSnapshot JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.TestDatanodeLayoutUpgrade   hadoop.hdfs.TestHFlush   hadoop.hdfs.qjournal.TestMiniJournalCluster   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.TestRefreshCallQueue   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   hadoop.hdfs.server.namenode.snapshot.TestSnapshot Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Issue HADOOP-12984 GITHUB PR https://github.com/apache/hadoop/pull/88 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 520e89b3f77f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a8d8b80 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9001/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Fixed tests,
          Pushed fix into PR.

          Show
          vinayrpet Vinayakumar B added a comment - Fixed tests, Pushed fix into PR.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 97 new or modified test files.
          0 mvndep 1m 4s Maven dependency ordering for branch
          +1 mvninstall 6m 49s trunk passed
          +1 compile 6m 0s trunk passed with JDK v1.8.0_74
          +1 compile 6m 44s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 5s trunk passed
          +1 mvnsite 2m 24s trunk passed
          +1 mvneclipse 0m 40s trunk passed
          +1 findbugs 5m 9s trunk passed
          +1 javadoc 2m 17s trunk passed with JDK v1.8.0_74
          +1 javadoc 3m 12s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 58s the patch passed
          +1 compile 6m 2s the patch passed with JDK v1.8.0_74
          +1 javac 8m 14s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 0 new + 737 unchanged - 1 fixed = 737 total (was 738)
          +1 javac 6m 2s root in the patch passed with JDK v1.8.0_74.
          +1 compile 6m 43s the patch passed with JDK v1.7.0_95
          +1 javac 14m 57s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735)
          +1 javac 6m 43s root in the patch passed with JDK v1.7.0_95.
          +1 checkstyle 1m 3s the patch passed
          +1 mvnsite 2m 21s the patch passed
          +1 mvneclipse 0m 39s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 5m 52s the patch passed
          +1 javadoc 2m 15s the patch passed with JDK v1.8.0_74
          +1 javadoc 3m 14s the patch passed with JDK v1.7.0_95
          -1 unit 6m 44s hadoop-common in the patch failed with JDK v1.8.0_74.
          +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_74.
          -1 unit 56m 42s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
          +1 unit 7m 14s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 59s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 55m 0s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 26s Patch does not generate ASF License warnings.
          195m 39s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.ha.TestZKFailoverController
            hadoop.TestRefreshCallQueue
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.TestHFlush
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
          JDK v1.7.0_95 Failed junit tests hadoop.TestRefreshCallQueue
            hadoop.hdfs.web.TestWebHDFS
            hadoop.hdfs.TestHFlush



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Issue HADOOP-12984
          GITHUB PR https://github.com/apache/hadoop/pull/88
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux cc81443e7e98 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 256c82f
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 97 new or modified test files. 0 mvndep 1m 4s Maven dependency ordering for branch +1 mvninstall 6m 49s trunk passed +1 compile 6m 0s trunk passed with JDK v1.8.0_74 +1 compile 6m 44s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 5s trunk passed +1 mvnsite 2m 24s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 5m 9s trunk passed +1 javadoc 2m 17s trunk passed with JDK v1.8.0_74 +1 javadoc 3m 12s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 58s the patch passed +1 compile 6m 2s the patch passed with JDK v1.8.0_74 +1 javac 8m 14s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 0 new + 737 unchanged - 1 fixed = 737 total (was 738) +1 javac 6m 2s root in the patch passed with JDK v1.8.0_74. +1 compile 6m 43s the patch passed with JDK v1.7.0_95 +1 javac 14m 57s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735) +1 javac 6m 43s root in the patch passed with JDK v1.7.0_95. +1 checkstyle 1m 3s the patch passed +1 mvnsite 2m 21s the patch passed +1 mvneclipse 0m 39s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 52s the patch passed +1 javadoc 2m 15s the patch passed with JDK v1.8.0_74 +1 javadoc 3m 14s the patch passed with JDK v1.7.0_95 -1 unit 6m 44s hadoop-common in the patch failed with JDK v1.8.0_74. +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_74. -1 unit 56m 42s hadoop-hdfs in the patch failed with JDK v1.8.0_74. +1 unit 7m 14s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 0m 59s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 55m 0s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 195m 39s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.TestRefreshCallQueue   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.TestHFlush   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits JDK v1.7.0_95 Failed junit tests hadoop.TestRefreshCallQueue   hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.TestHFlush Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Issue HADOOP-12984 GITHUB PR https://github.com/apache/hadoop/pull/88 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux cc81443e7e98 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 256c82f Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9005/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Test failures are unrelated.
          TestRefreshCallQueue has been fixed in HDFS-10253.
          TestHFlush has been tracked in HDFS-2043.
          Others are not related to patch.

          Show
          vinayrpet Vinayakumar B added a comment - Test failures are unrelated. TestRefreshCallQueue has been fixed in HDFS-10253 . TestHFlush has been tracked in HDFS-2043 . Others are not related to patch.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user vinayakumarb opened a pull request:

          https://github.com/apache/hadoop/pull/89

          HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for emporary directory in tests

          Rebased.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/vinayakumarb/hadoop features/HADOOP-12984

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/hadoop/pull/89.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #89


          commit ba071f9f0bb5b9133a0443653fa03f29b77e2624
          Author: Vinayakumar B <vinayakumarb@apache.org>
          Date: 2016-04-01T13:31:54Z

          HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for temporary directory in tests


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user vinayakumarb opened a pull request: https://github.com/apache/hadoop/pull/89 HADOOP-12984 . Add GenericTestUtils.getTestDir method and use it for emporary directory in tests Rebased. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vinayakumarb/hadoop features/ HADOOP-12984 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/89.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #89 commit ba071f9f0bb5b9133a0443653fa03f29b77e2624 Author: Vinayakumar B <vinayakumarb@apache.org> Date: 2016-04-01T13:31:54Z HADOOP-12984 . Add GenericTestUtils.getTestDir method and use it for temporary directory in tests
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vinayakumarb closed the pull request at:

          https://github.com/apache/hadoop/pull/88

          Show
          githubbot ASF GitHub Bot added a comment - Github user vinayakumarb closed the pull request at: https://github.com/apache/hadoop/pull/88
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aajisaka commented on a diff in the pull request:

          https://github.com/apache/hadoop/pull/89#discussion_r58334727

          — Diff: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDecommissioningStatus.java —
          @@ -87,7 +87,7 @@ public static void setUp() throws Exception {
          // Set up the hosts/exclude files.
          localFileSys = FileSystem.getLocal(conf);
          Path workingDir = localFileSys.getWorkingDirectory();

          • dir = new Path(workingDir, "build/test/data/work-dir/decommission");
            + dir = new Path(workingDir, "target/test/data" + "work-dir/decommission");
              • End diff –

          I'm thinking the string concatenation by `+` is unnecessarily.

          Show
          githubbot ASF GitHub Bot added a comment - Github user aajisaka commented on a diff in the pull request: https://github.com/apache/hadoop/pull/89#discussion_r58334727 — Diff: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDecommissioningStatus.java — @@ -87,7 +87,7 @@ public static void setUp() throws Exception { // Set up the hosts/exclude files. localFileSys = FileSystem.getLocal(conf); Path workingDir = localFileSys.getWorkingDirectory(); dir = new Path(workingDir, "build/test/data/work-dir/decommission"); + dir = new Path(workingDir, "target/test/data" + "work-dir/decommission"); End diff – I'm thinking the string concatenation by `+` is unnecessarily.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aajisaka commented on the pull request:

          https://github.com/apache/hadoop/pull/89#issuecomment-205168263

          Thank you for rebasing! Mostly looks good to me. Two comments:

          • Can we separate the patch into common change and hdfs change? That way the patch becomes smaller and the review becomes easier.
          • I'm thinking we should replace not only `test.build.data` but also `test.build.dir` with `GenericTestUtils.getTestDir`. I'm okay with doing this by separate jira(s).
          Show
          githubbot ASF GitHub Bot added a comment - Github user aajisaka commented on the pull request: https://github.com/apache/hadoop/pull/89#issuecomment-205168263 Thank you for rebasing! Mostly looks good to me. Two comments: Can we separate the patch into common change and hdfs change? That way the patch becomes smaller and the review becomes easier. I'm thinking we should replace not only `test.build.data` but also `test.build.dir` with `GenericTestUtils.getTestDir`. I'm okay with doing this by separate jira(s).
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 11m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 97 new or modified test files.
          0 mvndep 1m 32s Maven dependency ordering for branch
          +1 mvninstall 6m 38s trunk passed
          +1 compile 5m 37s trunk passed with JDK v1.8.0_77
          +1 compile 6m 32s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 4s trunk passed
          +1 mvnsite 2m 20s trunk passed
          +1 mvneclipse 0m 40s trunk passed
          +1 findbugs 5m 1s trunk passed
          +1 javadoc 2m 14s trunk passed with JDK v1.8.0_77
          +1 javadoc 3m 10s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 55s the patch passed
          +1 compile 5m 39s the patch passed with JDK v1.8.0_77
          +1 javac 7m 49s root-jdk1.8.0_77 with JDK v1.8.0_77 generated 0 new + 738 unchanged - 1 fixed = 738 total (was 739)
          +1 javac 5m 39s root in the patch passed with JDK v1.8.0_77.
          +1 compile 6m 33s the patch passed with JDK v1.7.0_95
          +1 javac 14m 22s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735)
          +1 javac 6m 33s root in the patch passed with JDK v1.7.0_95.
          +1 checkstyle 1m 3s the patch passed
          +1 mvnsite 2m 18s the patch passed
          +1 mvneclipse 0m 40s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 5m 45s the patch passed
          +1 javadoc 2m 17s the patch passed with JDK v1.8.0_77
          +1 javadoc 3m 16s the patch passed with JDK v1.7.0_95
          -1 unit 19m 49s hadoop-common in the patch failed with JDK v1.8.0_77.
          +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77.
          -1 unit 60m 28s hadoop-hdfs in the patch failed with JDK v1.8.0_77.
          +1 unit 8m 11s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 56m 7s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 30s Patch does not generate ASF License warnings.
          224m 42s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Issue HADOOP-12984
          GITHUB PR https://github.com/apache/hadoop/pull/89
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux bfd47ebba560 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1e6f929
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 97 new or modified test files. 0 mvndep 1m 32s Maven dependency ordering for branch +1 mvninstall 6m 38s trunk passed +1 compile 5m 37s trunk passed with JDK v1.8.0_77 +1 compile 6m 32s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 4s trunk passed +1 mvnsite 2m 20s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 5m 1s trunk passed +1 javadoc 2m 14s trunk passed with JDK v1.8.0_77 +1 javadoc 3m 10s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 55s the patch passed +1 compile 5m 39s the patch passed with JDK v1.8.0_77 +1 javac 7m 49s root-jdk1.8.0_77 with JDK v1.8.0_77 generated 0 new + 738 unchanged - 1 fixed = 738 total (was 739) +1 javac 5m 39s root in the patch passed with JDK v1.8.0_77. +1 compile 6m 33s the patch passed with JDK v1.7.0_95 +1 javac 14m 22s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735) +1 javac 6m 33s root in the patch passed with JDK v1.7.0_95. +1 checkstyle 1m 3s the patch passed +1 mvnsite 2m 18s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 45s the patch passed +1 javadoc 2m 17s the patch passed with JDK v1.8.0_77 +1 javadoc 3m 16s the patch passed with JDK v1.7.0_95 -1 unit 19m 49s hadoop-common in the patch failed with JDK v1.8.0_77. +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_77. -1 unit 60m 28s hadoop-hdfs in the patch failed with JDK v1.8.0_77. +1 unit 8m 11s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 56m 7s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 30s Patch does not generate ASF License warnings. 224m 42s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockManager JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Issue HADOOP-12984 GITHUB PR https://github.com/apache/hadoop/pull/89 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bfd47ebba560 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1e6f929 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9014/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Thanks Akira Ajisaka, I will separate the patch for Common and HDFS.
          I will push Common changes to same PR.
          I will raise a separate Jira for HDFS and Mapred/YARN changes.

          Show
          vinayrpet Vinayakumar B added a comment - Thanks Akira Ajisaka , I will separate the patch for Common and HDFS. I will push Common changes to same PR. I will raise a separate Jira for HDFS and Mapred/YARN changes.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vinayakumarb commented on the pull request:

          https://github.com/apache/hadoop/pull/89#issuecomment-205285404

          Rebased,
          and separated the common changes and hdfs changes.
          Pushed only common changes in this PR.

          Show
          githubbot ASF GitHub Bot added a comment - Github user vinayakumarb commented on the pull request: https://github.com/apache/hadoop/pull/89#issuecomment-205285404 Rebased, and separated the common changes and hdfs changes. Pushed only common changes in this PR.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Also, made changes to use GenericTestUtils methods for "test.build.dir" as well.

          Show
          vinayrpet Vinayakumar B added a comment - Also, made changes to use GenericTestUtils methods for "test.build.dir" as well.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 76 new or modified test files.
          +1 mvninstall 9m 50s trunk passed
          +1 compile 12m 23s trunk passed with JDK v1.8.0_74
          +1 compile 10m 46s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 1m 25s trunk passed
          +1 mvneclipse 0m 22s trunk passed
          +1 findbugs 2m 13s trunk passed
          +1 javadoc 1m 29s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 36s trunk passed with JDK v1.7.0_95
          +1 mvninstall 1m 2s the patch passed
          +1 compile 12m 23s the patch passed with JDK v1.8.0_74
          +1 javac 19m 10s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 0 new + 737 unchanged - 1 fixed = 737 total (was 738)
          +1 javac 12m 23s root in the patch passed with JDK v1.8.0_74.
          +1 compile 10m 54s the patch passed with JDK v1.7.0_95
          +1 javac 30m 5s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735)
          +1 javac 10m 54s root in the patch passed with JDK v1.7.0_95.
          +1 checkstyle 0m 31s the patch passed
          +1 mvnsite 1m 23s the patch passed
          +1 mvneclipse 0m 22s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 36s the patch passed
          +1 javadoc 1m 27s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 38s the patch passed with JDK v1.7.0_95
          -1 unit 10m 59s hadoop-common in the patch failed with JDK v1.8.0_74.
          +1 unit 10m 39s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 35s Patch does not generate ASF License warnings.
          97m 42s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.ipc.TestRPCWaitForProxy



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Issue HADOOP-12984
          GITHUB PR https://github.com/apache/hadoop/pull/89
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6a7597eede46 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 5092c94
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9018/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9018/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9018/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9018/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 76 new or modified test files. +1 mvninstall 9m 50s trunk passed +1 compile 12m 23s trunk passed with JDK v1.8.0_74 +1 compile 10m 46s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 25s trunk passed +1 mvneclipse 0m 22s trunk passed +1 findbugs 2m 13s trunk passed +1 javadoc 1m 29s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 36s trunk passed with JDK v1.7.0_95 +1 mvninstall 1m 2s the patch passed +1 compile 12m 23s the patch passed with JDK v1.8.0_74 +1 javac 19m 10s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 0 new + 737 unchanged - 1 fixed = 737 total (was 738) +1 javac 12m 23s root in the patch passed with JDK v1.8.0_74. +1 compile 10m 54s the patch passed with JDK v1.7.0_95 +1 javac 30m 5s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735) +1 javac 10m 54s root in the patch passed with JDK v1.7.0_95. +1 checkstyle 0m 31s the patch passed +1 mvnsite 1m 23s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 36s the patch passed +1 javadoc 1m 27s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 38s the patch passed with JDK v1.7.0_95 -1 unit 10m 59s hadoop-common in the patch failed with JDK v1.8.0_74. +1 unit 10m 39s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 35s Patch does not generate ASF License warnings. 97m 42s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.ipc.TestRPCWaitForProxy Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Issue HADOOP-12984 GITHUB PR https://github.com/apache/hadoop/pull/89 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6a7597eede46 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5092c94 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9018/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9018/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9018/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9018/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aajisaka commented on a diff in the pull request:

          https://github.com/apache/hadoop/pull/89#discussion_r58492524

          — Diff: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java —
          @@ -55,9 +56,11 @@
          public class TestFileUtil {
          private static final Log LOG = LogFactory.getLog(TestFileUtil.class);

          • private static final String TEST_ROOT_DIR = System.getProperty(
          • "test.build.data", "/tmp") + "/fu";
            + private static final String TEST_ROOT_DIR =
            + new File(GenericTestUtils.getTestDir(), "fu").getAbsolutePath();
            private static final File TEST_DIR = new File(TEST_ROOT_DIR);
            + private static final String cacheDir = System.getProperty("test.cache.data",
            + new File(TEST_DIR, "cache").getAbsolutePath());
              • End diff –
          • The code creates String by `File.getAbsolutePath()` and then creates File by `new File(String)`. It can be simplified by the following and `TEST_ROOT_DIR` can be removed.
            ```
            private static final File TEST_DIR = new File(GenericTestUtils.getTestDir(), "fu");
            ```
          • `cacheDir` is unused.
          Show
          githubbot ASF GitHub Bot added a comment - Github user aajisaka commented on a diff in the pull request: https://github.com/apache/hadoop/pull/89#discussion_r58492524 — Diff: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java — @@ -55,9 +56,11 @@ public class TestFileUtil { private static final Log LOG = LogFactory.getLog(TestFileUtil.class); private static final String TEST_ROOT_DIR = System.getProperty( "test.build.data", "/tmp") + "/fu"; + private static final String TEST_ROOT_DIR = + new File(GenericTestUtils.getTestDir(), "fu").getAbsolutePath(); private static final File TEST_DIR = new File(TEST_ROOT_DIR); + private static final String cacheDir = System.getProperty("test.cache.data", + new File(TEST_DIR, "cache").getAbsolutePath()); End diff – The code creates String by `File.getAbsolutePath()` and then creates File by `new File(String)`. It can be simplified by the following and `TEST_ROOT_DIR` can be removed. ``` private static final File TEST_DIR = new File(GenericTestUtils.getTestDir(), "fu"); ``` `cacheDir` is unused.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aajisaka commented on the pull request:

          https://github.com/apache/hadoop/pull/89#issuecomment-205680037

          Thank you for the update! I'm +1 if the above comments are addressed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user aajisaka commented on the pull request: https://github.com/apache/hadoop/pull/89#issuecomment-205680037 Thank you for the update! I'm +1 if the above comments are addressed.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Updated PR to fix above comment. please review.

          Show
          vinayrpet Vinayakumar B added a comment - Updated PR to fix above comment. please review.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 76 new or modified test files.
          +1 mvninstall 6m 42s trunk passed
          +1 compile 5m 55s trunk passed with JDK v1.8.0_77
          +1 compile 6m 47s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 57s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 35s trunk passed
          +1 javadoc 0m 57s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 6s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 42s the patch passed
          +1 compile 5m 49s the patch passed with JDK v1.8.0_77
          +1 javac 10m 26s root-jdk1.8.0_77 with JDK v1.8.0_77 generated 0 new + 737 unchanged - 1 fixed = 737 total (was 738)
          +1 javac 5m 49s root in the patch passed with JDK v1.8.0_77.
          +1 compile 6m 49s the patch passed with JDK v1.7.0_95
          +1 javac 17m 16s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735)
          +1 javac 6m 49s root in the patch passed with JDK v1.7.0_95.
          +1 checkstyle 0m 20s the patch passed
          +1 mvnsite 0m 58s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 49s the patch passed
          +1 javadoc 0m 51s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 6s the patch passed with JDK v1.7.0_95
          +1 unit 7m 13s hadoop-common in the patch passed with JDK v1.8.0_77.
          +1 unit 7m 25s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          59m 45s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Issue HADOOP-12984
          GITHUB PR https://github.com/apache/hadoop/pull/89
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux efecfb3a1403 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9174645
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9028/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9028/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 76 new or modified test files. +1 mvninstall 6m 42s trunk passed +1 compile 5m 55s trunk passed with JDK v1.8.0_77 +1 compile 6m 47s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 35s trunk passed +1 javadoc 0m 57s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 6s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 42s the patch passed +1 compile 5m 49s the patch passed with JDK v1.8.0_77 +1 javac 10m 26s root-jdk1.8.0_77 with JDK v1.8.0_77 generated 0 new + 737 unchanged - 1 fixed = 737 total (was 738) +1 javac 5m 49s root in the patch passed with JDK v1.8.0_77. +1 compile 6m 49s the patch passed with JDK v1.7.0_95 +1 javac 17m 16s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 0 new + 734 unchanged - 1 fixed = 734 total (was 735) +1 javac 6m 49s root in the patch passed with JDK v1.7.0_95. +1 checkstyle 0m 20s the patch passed +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 49s the patch passed +1 javadoc 0m 51s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 6s the patch passed with JDK v1.7.0_95 +1 unit 7m 13s hadoop-common in the patch passed with JDK v1.8.0_77. +1 unit 7m 25s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 59m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Issue HADOOP-12984 GITHUB PR https://github.com/apache/hadoop/pull/89 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux efecfb3a1403 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9174645 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9028/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9028/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank you for the update, Vinayakumar B. +1 for the latest patch. Would you rebase the patch for branch-2?

          Show
          ajisakaa Akira Ajisaka added a comment - Thank you for the update, Vinayakumar B . +1 for the latest patch. Would you rebase the patch for branch-2?
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user vinayakumarb opened a pull request:

          https://github.com/apache/hadoop/pull/91

          HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for temporary directory in tests (branch-2)

          Rebased against branch-2

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/vinayakumarb/hadoop features/HADOOP-12984-branch-2

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/hadoop/pull/91.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #91


          commit c9675a22a320ecaa0dcb6a445b0d259aa91baa22
          Author: Vinayakumar B <vinayakumarb@apache.org>
          Date: 2016-04-05T14:29:16Z

          HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for emporary directory in tests


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user vinayakumarb opened a pull request: https://github.com/apache/hadoop/pull/91 HADOOP-12984 . Add GenericTestUtils.getTestDir method and use it for temporary directory in tests (branch-2) Rebased against branch-2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/vinayakumarb/hadoop features/ HADOOP-12984 -branch-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/91.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #91 commit c9675a22a320ecaa0dcb6a445b0d259aa91baa22 Author: Vinayakumar B <vinayakumarb@apache.org> Date: 2016-04-05T14:29:16Z HADOOP-12984 . Add GenericTestUtils.getTestDir method and use it for emporary directory in tests
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1, thanks a lot!

          Show
          ajisakaa Akira Ajisaka added a comment - +1, thanks a lot!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/hadoop/pull/89

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/hadoop/pull/89
          Hide
          vinayrpet Vinayakumar B added a comment -

          Committed to trunk and branch-2.

          Thanks Steve Loughran for the initial patch.
          Thanks Chris Nauroth and Akira Ajisaka for great reviews.

          Show
          vinayrpet Vinayakumar B added a comment - Committed to trunk and branch-2. Thanks Steve Loughran for the initial patch. Thanks Chris Nauroth and Akira Ajisaka for great reviews.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vinayakumarb closed the pull request at:

          https://github.com/apache/hadoop/pull/91

          Show
          githubbot ASF GitHub Bot added a comment - Github user vinayakumarb closed the pull request at: https://github.com/apache/hadoop/pull/91
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9574 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9574/)
          HADOOP-12984. Add GenericTestUtils.getTestDir method and use it for (vinayakumarb: rev 8d29e2451f5ca60f864c7ece16722c0abdd1c657)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestAuthenticationSessionCookie.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestPathData.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/sink/RollingFileSystemSinkTestBase.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/KeyStoreTestUtil.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFileAppend.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/ClientBaseWithFixes.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredShell.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellTouch.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileSeek.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestGetFileBlockLocations.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileUnsortedByteArrays.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileSeqFileComparison.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileStreams.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFileSerialization.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/JarFinder.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMapping.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestTextCommand.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestRunJar.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestCredentials.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileContextMainOperationsBaseTest.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestKeyShell.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewfsFileStatus.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestSharedFileDescriptorFactory.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestBloomMapFile.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileContextURIBase.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestChecksumFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestZKUtil.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredentialProviderFactory.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerLifecycle.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestListFiles.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestSSLHttpServer.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/sftp/TestSFTPFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FSTestWrapper.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/TestCryptoStreamsForLocalFS.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTrash.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestJarFinder.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestClasspath.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/TestReloadingX509TrustManager.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileContextTestHelper.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemTestHelper.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFile.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileByteArrays.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestDU.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestPath.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/TestSSLFactory.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileSplit.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSetFile.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTruncatedInputBug.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestArrayFile.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileComparator2.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystemBasics.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestMapFile.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestVLong.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFileSync.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestHostsFileReader.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpCookieFlag.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestDFVariations.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileComparators.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCodec.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestApplicationClassLoader.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestAvroFSInput.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystemPermission.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHardLink.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileContextResolveAfs.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9574 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9574/ ) HADOOP-12984 . Add GenericTestUtils.getTestDir method and use it for (vinayakumarb: rev 8d29e2451f5ca60f864c7ece16722c0abdd1c657) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestAuthenticationSessionCookie.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestPathData.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/sink/RollingFileSystemSinkTestBase.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/KeyStoreTestUtil.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFileAppend.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ha/ClientBaseWithFixes.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredShell.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellTouch.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestGenericOptionsParser.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileSeek.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestGetFileBlockLocations.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileUnsortedByteArrays.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileSeqFileComparison.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileStreams.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFileSerialization.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/JarFinder.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMapping.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestTextCommand.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestRunJar.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestCredentials.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileContextMainOperationsBaseTest.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestWinUtils.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestKeyShell.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewfsFileStatus.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestSharedFileDescriptorFactory.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestBloomMapFile.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileContextURIBase.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestChecksumFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestZKUtil.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredentialProviderFactory.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerLifecycle.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestListFiles.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestSSLHttpServer.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/sftp/TestSFTPFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FSTestWrapper.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/TestCryptoStreamsForLocalFS.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTrash.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestJarFinder.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestClasspath.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/TestReloadingX509TrustManager.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileContextTestHelper.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemTestHelper.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFile.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileByteArrays.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestDU.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestPath.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellCopy.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/TestSSLFactory.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileSplit.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSetFile.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTruncatedInputBug.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestArrayFile.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileComparator2.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystemBasics.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestMapFile.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestVLong.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFileSync.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestHostsFileReader.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpCookieFlag.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestDFVariations.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/file/tfile/TestTFileComparators.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCodec.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestApplicationClassLoader.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestAvroFSInput.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystemPermission.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHardLink.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileContextResolveAfs.java
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          There is a complication error in branch-2

          [ERROR] COMPILATION ERROR : 
          [INFO] -------------------------------------------------------------
          [ERROR] /Users/szetszwo/hadoop/b-2/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java:[569,31] cannot find symbol
            symbol:   variable TEST_ROOT_DIR
            location: class org.apache.hadoop.fs.TestFileUtil
          [ERROR] /Users/szetszwo/hadoop/b-2/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java:[570,31] cannot find symbol
            symbol:   variable TEST_ROOT_DIR
            location: class org.apache.hadoop.fs.TestFileUtil
          [INFO] 2 errors 
          
          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - There is a complication error in branch-2 [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /Users/szetszwo/hadoop/b-2/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java:[569,31] cannot find symbol symbol: variable TEST_ROOT_DIR location: class org.apache.hadoop.fs.TestFileUtil [ERROR] /Users/szetszwo/hadoop/b-2/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java:[570,31] cannot find symbol symbol: variable TEST_ROOT_DIR location: class org.apache.hadoop.fs.TestFileUtil [INFO] 2 errors
          Hide
          vinayrpet Vinayakumar B added a comment -

          Thanks Tsz Wo Nicholas Sze, I have pushed addendum commit to fix the compilation.

          Show
          vinayrpet Vinayakumar B added a comment - Thanks Tsz Wo Nicholas Sze , I have pushed addendum commit to fix the compilation.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          No problem. Below is a suggested fix.

          diff --git a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          index c478681..76ccb75 100644
          --- a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          +++ b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java
          @@ -566,8 +566,8 @@ private boolean copyMerge(String src, String dst)
               final boolean result;
           
               try {
          -      Path srcPath = new Path(TEST_ROOT_DIR, src);
          -      Path dstPath = new Path(TEST_ROOT_DIR, dst);
          +      Path srcPath = new Path(TEST_DIR.toString(), src);
          +      Path dstPath = new Path(TEST_DIR.toString(), dst);
                 boolean deleteSource = false;
                 String addString = null;
                 result = FileUtil.copyMerge(fs, srcPath, fs, dstPath, deleteSource, conf,
          
          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - No problem. Below is a suggested fix. diff --git a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java index c478681..76ccb75 100644 --- a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java +++ b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java @@ -566,8 +566,8 @@ private boolean copyMerge( String src, String dst) final boolean result; try { - Path srcPath = new Path(TEST_ROOT_DIR, src); - Path dstPath = new Path(TEST_ROOT_DIR, dst); + Path srcPath = new Path(TEST_DIR.toString(), src); + Path dstPath = new Path(TEST_DIR.toString(), dst); boolean deleteSource = false ; String addString = null ; result = FileUtil.copyMerge(fs, srcPath, fs, dstPath, deleteSource, conf,
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Nicholas and Vinayakumar! I just prepared the same addendum patch. My late +1.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Nicholas and Vinayakumar! I just prepared the same addendum patch. My late +1.
          Hide
          vinayrpet Vinayakumar B added a comment -

          I have already pushed below one.

               try {
          -      Path srcPath = new Path(TEST_ROOT_DIR, src);
          -      Path dstPath = new Path(TEST_ROOT_DIR, dst);
          +      Path srcPath = new Path(TEST_DIR.getAbsolutePath(), src);
          +      Path dstPath = new Path(TEST_DIR.getAbsolutePath(), dst);
                 boolean deleteSource = false;
                 String addString = null;
                 result = FileUtil.copyMerge(fs, srcPath, fs, dstPath, deleteSource, conf,
          
          Show
          vinayrpet Vinayakumar B added a comment - I have already pushed below one. try { - Path srcPath = new Path(TEST_ROOT_DIR, src); - Path dstPath = new Path(TEST_ROOT_DIR, dst); + Path srcPath = new Path(TEST_DIR.getAbsolutePath(), src); + Path dstPath = new Path(TEST_DIR.getAbsolutePath(), dst); boolean deleteSource = false ; String addString = null ; result = FileUtil.copyMerge(fs, srcPath, fs, dstPath, deleteSource, conf,
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          That's great. Thanks!

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - That's great. Thanks!
          Hide
          xyao Xiaoyu Yao added a comment - - edited

          Propose to port this along with HDFS-10256 to branch-2.8. The cherry-pick is mostly clean with only few imports adjustment. RollingFileSystemSinkTestBase.java is removed as RollingFileSystemSink does not existing in branch-2.8.

          Below is a complete list of conflicts, I will post the branch-2.8 patch. Steve Loughran, can you help review?

          	both modified:   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestKeyShell.java
          	both modified:   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          	both modified:   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTrash.java
          	deleted by us:   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/sink/RollingFileSystemSinkTestBase.java
          	both modified:   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMapping.java
          	both modified:   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredShell.java
          	both modified:   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/TestReloadingX509TrustManager.java
          
          Show
          xyao Xiaoyu Yao added a comment - - edited Propose to port this along with HDFS-10256 to branch-2.8. The cherry-pick is mostly clean with only few imports adjustment. RollingFileSystemSinkTestBase.java is removed as RollingFileSystemSink does not existing in branch-2.8. Below is a complete list of conflicts, I will post the branch-2.8 patch. Steve Loughran , can you help review? both modified: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestKeyShell.java both modified: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java both modified: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestTrash.java deleted by us: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/sink/RollingFileSystemSinkTestBase.java both modified: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMapping.java both modified: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/alias/TestCredShell.java both modified: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/ssl/TestReloadingX509TrustManager.java
          Hide
          liuml07 Mingliang Liu added a comment -

          +1 for the back port. Thanks Xiaoyu Yao for taking care of this.

          Show
          liuml07 Mingliang Liu added a comment - +1 for the back port. Thanks Xiaoyu Yao for taking care of this.
          Hide
          liuml07 Mingliang Liu added a comment -

          +1 for HADOOP-12984.branch-2.8.00.patch. Thanks Xiaoyu.

          Show
          liuml07 Mingliang Liu added a comment - +1 for HADOOP-12984.branch-2.8.00.patch . Thanks Xiaoyu.
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Mingliang Liu for the review. I've cherry-pick the change to branch-2.8.

          Show
          xyao Xiaoyu Yao added a comment - Thanks Mingliang Liu for the review. I've cherry-pick the change to branch-2.8.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user steveloughran closed the pull request at:

          https://github.com/apache/hadoop/pull/53

          Show
          githubbot ASF GitHub Bot added a comment - Github user steveloughran closed the pull request at: https://github.com/apache/hadoop/pull/53

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development