Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12415

hdfs and nfs builds broken on -missing compile-time dependency on netty

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: nfs
    • Labels:
      None
    • Environment:

      Bigtop, plain Linux distro of any kind

      Description

      As discovered in BIGTOP-2049 hadoop-nfs module compilation is broken. Looks like that HADOOP-11489 is the root-cause of it.

        Issue Links

          Activity

          Hide
          cos Konstantin Boudnik added a comment -

          A fix is proposed here

          Show
          cos Konstantin Boudnik added a comment - A fix is proposed here
          Hide
          stevel@apache.org Steve Loughran added a comment -

          ..Konstantin, can you attach the patch here for Hadoop's jenkins test run to look at?

          Show
          stevel@apache.org Steve Loughran added a comment - ..Konstantin, can you attach the patch here for Hadoop's jenkins test run to look at?
          Hide
          tomzeng Tom Zeng added a comment -

          Steve Loughran patch attached

          Show
          tomzeng Tom Zeng added a comment - Steve Loughran patch attached
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 32s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 52s There were no new javac warning messages.
          +1 javadoc 10m 8s There were no new javadoc warning messages.
          +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 27s mvn install still works.
          +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
          +1 common tests 22m 58s Tests passed in hadoop-common.
              58m 57s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12756267/HADOOP-12415.patch
          Optional Tests javadoc javac unit
          git revision trunk / bf2f2b4
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7669/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7669/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7669/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 32s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 52s There were no new javac warning messages. +1 javadoc 10m 8s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. +1 common tests 22m 58s Tests passed in hadoop-common.     58m 57s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12756267/HADOOP-12415.patch Optional Tests javadoc javac unit git revision trunk / bf2f2b4 hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7669/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7669/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7669/console This message was automatically generated.
          Hide
          tomzeng Tom Zeng added a comment -

          Steve Loughran the patch is to the hadoop common pom file, no new tests needed, as long as existing tests pass.

          Show
          tomzeng Tom Zeng added a comment - Steve Loughran the patch is to the hadoop common pom file, no new tests needed, as long as existing tests pass.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I don't want to add a new "compile" scope JAR to hadoop-common, as it only causes classpath pain downstream.

          If this is just for hadoop-nfs module, then shouldn't it go in there? As it will be needed at runtime, presumably

          Show
          stevel@apache.org Steve Loughran added a comment - I don't want to add a new "compile" scope JAR to hadoop-common, as it only causes classpath pain downstream. If this is just for hadoop-nfs module, then shouldn't it go in there? As it will be needed at runtime, presumably
          Hide
          tomzeng Tom Zeng added a comment -

          It wasn't enough to just add it to hadoop-nfs, hadoop-hdfs will still fail, that's why I move it up to hadoop-common. We verified it's working for BIGTOP-2049

          Show
          tomzeng Tom Zeng added a comment - It wasn't enough to just add it to hadoop-nfs, hadoop-hdfs will still fail, that's why I move it up to hadoop-common. We verified it's working for BIGTOP-2049
          Hide
          tomzeng Tom Zeng added a comment -

          Adding the io.netty dependency to only hadoop-nfs gets the following failure:
          ...
          [INFO] Apache Hadoop Common ............................... SUCCESS [01:37 min]
          [INFO] Apache Hadoop NFS .................................. SUCCESS [ 7.346 s]
          [INFO] Apache Hadoop KMS .................................. SUCCESS [ 17.295 s]
          [INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.135 s]
          [INFO] Apache Hadoop HDFS ................................. FAILURE [ 29.510 s]
          [INFO] Apache Hadoop HttpFS ............................... SKIPPED

          Show
          tomzeng Tom Zeng added a comment - Adding the io.netty dependency to only hadoop-nfs gets the following failure: ... [INFO] Apache Hadoop Common ............................... SUCCESS [01:37 min] [INFO] Apache Hadoop NFS .................................. SUCCESS [ 7.346 s] [INFO] Apache Hadoop KMS .................................. SUCCESS [ 17.295 s] [INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.135 s] [INFO] Apache Hadoop HDFS ................................. FAILURE [ 29.510 s] [INFO] Apache Hadoop HttpFS ............................... SKIPPED
          Hide
          cos Konstantin Boudnik added a comment -

          +1 the patch looks correct and the placement of the dependency is right. If anything HADOOP-11489 has introduced the bug. I will commit this in a bit to the trunk and branch-2. Thanks for the patch, Tom Zeng!

          Show
          cos Konstantin Boudnik added a comment - +1 the patch looks correct and the placement of the dependency is right. If anything HADOOP-11489 has introduced the bug. I will commit this in a bit to the trunk and branch-2. Thanks for the patch, Tom Zeng !
          Hide
          andrew.wang Andrew Wang added a comment -

          -1 for now, please hold off on committing Cos. I'd like to spend some more time figuring out what's happening, because we need to be super careful about adding deps to hadoop-common.

          hadoop-nfs builds with current trunk for me and I assume Hadoop precommit since it passed on HADOOP-11489. Do we know why exactly it doesn't work for Bigtop? Also if this was broken by HADOOP-11489, why isn't the fix simply reverting HADOOP-11489?

          Show
          andrew.wang Andrew Wang added a comment - -1 for now, please hold off on committing Cos. I'd like to spend some more time figuring out what's happening, because we need to be super careful about adding deps to hadoop-common. hadoop-nfs builds with current trunk for me and I assume Hadoop precommit since it passed on HADOOP-11489 . Do we know why exactly it doesn't work for Bigtop? Also if this was broken by HADOOP-11489 , why isn't the fix simply reverting HADOOP-11489 ?
          Hide
          cos Konstantin Boudnik added a comment -

          Andrew, please follow the discussion over here BIGTOP-2049. test-patch isn't really the golden standard when it comes to the testing of the depedency-related changes, as it most likely has pre-populated maven cache.

          Show
          cos Konstantin Boudnik added a comment - Andrew, please follow the discussion over here BIGTOP-2049 . test-patch isn't really the golden standard when it comes to the testing of the depedency-related changes, as it most likely has pre-populated maven cache.
          Hide
          andrew.wang Andrew Wang added a comment -

          Hey Cos, I just tried running some mvn commands like this for a fresh cache, build still worked:

          cd hadoop-common-project/hadoop-nfs
          mvn clean package -DskipTests -Dmaven.repo.local=/tmp/hadoop-12415-hadoop-nfs/
          
          cd hadoop-hdfs-project/hadoop-hdfs-nfs
          mvn clean package -DskipTests -Dmaven.repo.local=/tmp/hadoop-12415/
          

          Could you post some standalone steps to reproduce this issue?

          Show
          andrew.wang Andrew Wang added a comment - Hey Cos, I just tried running some mvn commands like this for a fresh cache, build still worked: cd hadoop-common-project/hadoop-nfs mvn clean package -DskipTests -Dmaven.repo.local=/tmp/hadoop-12415-hadoop-nfs/ cd hadoop-hdfs-project/hadoop-hdfs-nfs mvn clean package -DskipTests -Dmaven.repo.local=/tmp/hadoop-12415/ Could you post some standalone steps to reproduce this issue?
          Hide
          tomzeng Tom Zeng added a comment -

          Andrew Wang can you checkout branch-2.7.1 and build Zookeeper first and then Hadoop?

          Show
          tomzeng Tom Zeng added a comment - Andrew Wang can you checkout branch-2.7.1 and build Zookeeper first and then Hadoop?
          Hide
          tomzeng Tom Zeng added a comment -

          And build at the top level, I can reproduce it every time (within or outside of Bigtop build).

          Show
          tomzeng Tom Zeng added a comment - And build at the top level, I can reproduce it every time (within or outside of Bigtop build).
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I'm a -1 too, as I don't want to add things to hadoop-common's dependencies, not when they should be server-side only.

          HDFS does import netty-all, so why HDFS is failing to build here is odd. That said, we do appear to be referencing > 1 netty library in hadoop-project:

                <dependency>
                  <groupId>io.netty</groupId>
                  <artifactId>netty</artifactId>
                  <version>3.6.2.Final</version>
                </dependency>
          
                <dependency>
                  <groupId>io.netty</groupId>
                  <artifactId>netty-all</artifactId>
                  <version>4.1.0.Beta5</version>
                </dependency>
          

          this looks "odd"

          Show
          stevel@apache.org Steve Loughran added a comment - I'm a -1 too, as I don't want to add things to hadoop-common's dependencies, not when they should be server-side only. HDFS does import netty-all, so why HDFS is failing to build here is odd. That said, we do appear to be referencing > 1 netty library in hadoop-project: <dependency> <groupId>io.netty</groupId> <artifactId>netty</artifactId> <version>3.6.2.Final</version> </dependency> <dependency> <groupId>io.netty</groupId> <artifactId>netty-all</artifactId> <version>4.1.0.Beta5</version> </dependency> this looks "odd"
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Tom Zeng -are you using a later/private ZK build? Maybe that's the problem: hadoop has been picking up netty from there, and if it changes its dependencies, then things stop.

          If so, we should fix hadoop-hdfs and hadoop-nfs, not common, because anything in common ends up the classpath of everything downstream, client or server-side

          Show
          stevel@apache.org Steve Loughran added a comment - Tom Zeng -are you using a later/private ZK build? Maybe that's the problem: hadoop has been picking up netty from there, and if it changes its dependencies, then things stop. If so, we should fix hadoop-hdfs and hadoop-nfs, not common, because anything in common ends up the classpath of everything downstream, client or server-side
          Hide
          andrew.wang Andrew Wang added a comment -

          Right, I note that in the hadoop-common/pom.xml we do some excludes related to ZK and netty, it could be related to that?

          <dependency>
                <groupId>org.apache.zookeeper</groupId>
                <artifactId>zookeeper</artifactId>
                <exclusions>
          ...
                  <exclusion>
                    <groupId>org.jboss.netty</groupId>
                    <artifactId>netty</artifactId>
                  </exclusion>
          ...
              </dependency>
          
          Show
          andrew.wang Andrew Wang added a comment - Right, I note that in the hadoop-common/pom.xml we do some excludes related to ZK and netty, it could be related to that? <dependency> <groupId>org.apache.zookeeper</groupId> <artifactId>zookeeper</artifactId> <exclusions> ... <exclusion> <groupId>org.jboss.netty</groupId> <artifactId>netty</artifactId> </exclusion> ... </dependency>
          Hide
          cos Konstantin Boudnik added a comment -

          Might be a red-herring, because we surely didn't observe the problem in 2.6 and this exclusion has been sitting in for 2+ years.

          Show
          cos Konstantin Boudnik added a comment - Might be a red-herring, because we surely didn't observe the problem in 2.6 and this exclusion has been sitting in for 2+ years.
          Hide
          tomzeng Tom Zeng added a comment -

          Yeah, that didn't seem to be the cause, removing the exclusion didn't work. I found that adding the io.netty dependency in both the hadoop-common/hadoop-nfs pom and the hadoop-hdfs pom worked. And in hadoop-hdfs both io.netty and io.netty-all are needed, kinda odd. If you guys think this is better (does not change hadoop-common), I will go ahead and submit a patch for it.

          Show
          tomzeng Tom Zeng added a comment - Yeah, that didn't seem to be the cause, removing the exclusion didn't work. I found that adding the io.netty dependency in both the hadoop-common/hadoop-nfs pom and the hadoop-hdfs pom worked. And in hadoop-hdfs both io.netty and io.netty-all are needed, kinda odd. If you guys think this is better (does not change hadoop-common), I will go ahead and submit a patch for it.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Tom. yes, please do it for hdfs and nfs -and for hdfs tag as provided. That way it stays off the client classpaths, which is the source of most of the transitive suffering

          Show
          stevel@apache.org Steve Loughran added a comment - Tom. yes, please do it for hdfs and nfs -and for hdfs tag as provided. That way it stays off the client classpaths, which is the source of most of the transitive suffering
          Hide
          cos Konstantin Boudnik added a comment -

          Steve, the bug isn't Bigtop specific. As any other integration bug it is easily found with bigtop, but one can run the build from 2.7.1 release and step on the same issue. Hence, renaming the bug again.

          Show
          cos Konstantin Boudnik added a comment - Steve, the bug isn't Bigtop specific. As any other integration bug it is easily found with bigtop, but one can run the build from 2.7.1 release and step on the same issue. Hence, renaming the bug again.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          OK...I wasn't blaming bigtop, just emphasising that the likely cause was the integration building picking up different published ZK dependencies.

          Show
          stevel@apache.org Steve Loughran added a comment - OK...I wasn't blaming bigtop, just emphasising that the likely cause was the integration building picking up different published ZK dependencies.
          Hide
          cos Konstantin Boudnik added a comment -

          Don't worry - I haven't said anything about 'blaming'. I simply stated that the issue can be reproduced in any environment.

          Show
          cos Konstantin Boudnik added a comment - Don't worry - I haven't said anything about 'blaming'. I simply stated that the issue can be reproduced in any environment.
          Hide
          tomzeng Tom Zeng added a comment -

          New patch, update hadoop-common-project/hadoop-nfs/pom.xml and /hadoop-hdfs-project/hadoop-hdfs/pom.xml instead of hadoop-common pom

          Show
          tomzeng Tom Zeng added a comment - New patch, update hadoop-common-project/hadoop-nfs/pom.xml and /hadoop-hdfs-project/hadoop-hdfs/pom.xml instead of hadoop-common pom
          Hide
          tomzeng Tom Zeng added a comment -

          Steve Loughran new patch attached that updates hadoop-nfs and hadoop-hdfs instead of hadoop-common pom.

          Show
          tomzeng Tom Zeng added a comment - Steve Loughran new patch attached that updates hadoop-nfs and hadoop-hdfs instead of hadoop-common pom.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 49s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 9m 5s There were no new javac warning messages.
          +1 javadoc 13m 48s There were no new javadoc warning messages.
          +1 release audit 0m 30s The applied patch does not increase the total number of release audit warnings.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 2m 5s mvn install still works.
          +1 eclipse:eclipse 0m 44s The patch built with eclipse:eclipse.
          +1 common tests 0m 27s Tests passed in hadoop-nfs.
          -1 hdfs tests 83m 39s Tests failed in hadoop-hdfs.
              128m 12s  



          Reason Tests
          Failed unit tests hadoop.hdfs.TestWriteRead
            hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
            hadoop.hdfs.server.namenode.ha.TestDNFencing
            hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
            hadoop.hdfs.TestHFlush
            hadoop.security.TestPermission
            hadoop.hdfs.TestParallelRead
            hadoop.fs.viewfs.TestViewFsHdfs
            hadoop.hdfs.TestBlockReaderLocalLegacy
            hadoop.fs.TestHdfsNativeCodeLoader
            hadoop.hdfs.server.namenode.TestAuditLogger
            hadoop.hdfs.server.namenode.TestXAttrConfigFlag
            hadoop.hdfs.TestPread
            hadoop.hdfs.TestMiniDFSCluster
            hadoop.hdfs.server.namenode.TestTransferFsImage
            hadoop.hdfs.TestWriteConfigurationToDFS
            hadoop.hdfs.TestDatanodeRegistration
            hadoop.hdfs.web.TestWebHDFSXAttr
            hadoop.hdfs.TestDFSRollback
            hadoop.hdfs.TestDataTransferKeepalive
            hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA
            hadoop.hdfs.TestDatanodeConfig
            hadoop.hdfs.TestDFSFinalize
            hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA
            hadoop.fs.TestWebHdfsFileContextMainOperations
            hadoop.hdfs.server.namenode.TestCommitBlockSynchronization
            hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
            hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
            hadoop.fs.TestGlobPaths
            hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
            hadoop.hdfs.server.namenode.ha.TestHASafeMode
            hadoop.hdfs.TestDFSShell
            hadoop.hdfs.tools.TestDFSHAAdminMiniCluster
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.TestSeekBug
            hadoop.fs.loadGenerator.TestLoadGenerator
            hadoop.hdfs.TestCrcCorruption
            hadoop.fs.contract.hdfs.TestHDFSContractMkdir
            hadoop.hdfs.server.namenode.TestStorageRestore
            hadoop.hdfs.TestAbandonBlock
            hadoop.hdfs.TestGetFileChecksum
            hadoop.hdfs.server.namenode.TestBlockUnderConstruction
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary
            hadoop.hdfs.TestFileCreationDelete
            hadoop.hdfs.TestReadWhileWriting
            hadoop.fs.viewfs.TestViewFileSystemWithAcls
            hadoop.fs.contract.hdfs.TestHDFSContractConcat
            hadoop.hdfs.util.TestDiff
            hadoop.hdfs.security.TestDelegationToken
            hadoop.fs.TestSymlinkHdfsDisable
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory
            hadoop.hdfs.TestMissingBlocksAlert
            hadoop.hdfs.TestBlocksScheduledCounter
            hadoop.hdfs.TestSmallBlock
            hadoop.cli.TestDeleteCLI
            hadoop.hdfs.TestDFSClientRetries
            hadoop.fs.viewfs.TestViewFsWithXAttrs
            hadoop.hdfs.TestDFSMkdirs
            hadoop.hdfs.tools.TestDFSAdmin
            hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.server.namenode.TestNameNodeMXBean
            hadoop.hdfs.web.TestWebHDFSForHA
            hadoop.hdfs.tools.TestGetConf
            hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
            hadoop.fs.viewfs.TestViewFsDefaultValue
            hadoop.fs.contract.hdfs.TestHDFSContractOpen
            hadoop.hdfs.server.namenode.TestDeleteRace
            hadoop.hdfs.TestFSInputChecker
            hadoop.hdfs.web.TestWebHdfsWithAuthenticationFilter
            hadoop.fs.contract.hdfs.TestHDFSContractRename
            hadoop.hdfs.web.TestWebHdfsTimeouts
            hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA
            hadoop.hdfs.server.namenode.ha.TestHAMetrics
            hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler
            hadoop.hdfs.TestRollingUpgradeRollback
            hadoop.hdfs.TestRemoteBlockReader
            hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
            hadoop.hdfs.tools.TestDFSAdminWithHA
            hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.TestLeaseRecovery
            hadoop.fs.viewfs.TestViewFsAtHdfsRoot
            hadoop.hdfs.server.namenode.TestBackupNode
            hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus
            hadoop.cli.TestCryptoAdminCLI
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForXAttr
            hadoop.hdfs.tools.TestDebugAdmin
            hadoop.security.TestRefreshUserMappings
            hadoop.hdfs.TestLargeBlock
            hadoop.hdfs.server.namenode.TestFSNamesystemMBean
            hadoop.fs.viewfs.TestViewFileSystemWithXAttrs
            hadoop.hdfs.tools.TestDFSZKFailoverController
            hadoop.hdfs.server.namenode.ha.TestNNHealthCheck
            hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.TestListFilesInFileContext
            hadoop.hdfs.tools.TestDelegationTokenFetcher
            hadoop.fs.TestFcHdfsSetUMask
            hadoop.hdfs.server.namenode.TestNameNodeRecovery
            hadoop.hdfs.TestDatanodeReport
            hadoop.hdfs.TestFileAppend2
            hadoop.fs.shell.TestHdfsTextCommand
            hadoop.hdfs.TestFsShellPermission
            hadoop.hdfs.TestHDFSServerPorts
            hadoop.hdfs.server.namenode.ha.TestHAFsck
            hadoop.hdfs.TestRollingUpgradeDowngrade
            hadoop.hdfs.TestFileLengthOnClusterRestart
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing
            hadoop.TestGenericRefresh
            hadoop.hdfs.TestParallelUnixDomainRead
            hadoop.hdfs.TestDFSUtil
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication
            hadoop.fs.TestSymlinkHdfsFileSystem
            hadoop.hdfs.TestGetBlocks
            hadoop.hdfs.security.token.block.TestBlockToken
            hadoop.fs.contract.hdfs.TestHDFSContractAppend
            hadoop.fs.contract.hdfs.TestHDFSContractDelete
            hadoop.hdfs.web.TestWebHdfsTokens
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.TestEncryptionZonesWithKMS
            hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
            hadoop.hdfs.TestClientReportBadBlock
            hadoop.hdfs.TestDFSStorageStateRecovery
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
            hadoop.cli.TestHDFSCLI
            hadoop.fs.TestSWebHdfsFileContextMainOperations
            hadoop.hdfs.TestRestartDFS
            hadoop.hdfs.TestFileAppend4
            hadoop.hdfs.web.TestWebHdfsUrl
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
            hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover
            hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
            hadoop.hdfs.TestSetTimes
            hadoop.hdfs.server.namenode.TestAclConfigFlag
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.namenode.TestNameNodeAcl
            hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy
            hadoop.hdfs.server.namenode.TestSecondaryWebUi
            hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.fs.contract.hdfs.TestHDFSContractSeek
            hadoop.hdfs.TestSetrepIncreasing
            hadoop.fs.viewfs.TestViewFsWithAcls
            hadoop.hdfs.TestLease
            hadoop.hdfs.TestSafeMode
            hadoop.hdfs.server.namenode.TestEditLogAutoroll
            hadoop.hdfs.TestDFSUpgrade
            hadoop.hdfs.TestDecommission
            hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
            hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
            hadoop.fs.TestHDFSFileContextMainOperations
            hadoop.hdfs.TestFileCorruption
            hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs
            hadoop.hdfs.TestQuota
            hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA
            hadoop.hdfs.TestFileStatus
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename
            hadoop.hdfs.server.namenode.ha.TestHAConfiguration
            hadoop.fs.TestResolveHdfsSymlink
            hadoop.fs.TestSymlinkHdfsFileContext
            hadoop.hdfs.TestEncryptionZonesWithHA
            hadoop.hdfs.server.namenode.TestDeadDatanode
            hadoop.cli.TestXAttrCLI
            hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
            hadoop.fs.TestUrlStreamHandler
            hadoop.hdfs.TestClose
            hadoop.hdfs.TestMultiThreadedHflush
            hadoop.hdfs.TestDFSClientExcludedNodes
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
            hadoop.hdfs.server.namenode.TestEditLogRace
            hadoop.hdfs.TestFileConcurrentReader
            hadoop.hdfs.TestClientBlockVerification
            hadoop.security.TestPermissionSymlinks
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            hadoop.cli.TestAclCLI
            hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
            hadoop.hdfs.TestDataTransferProtocol
            hadoop.hdfs.server.namenode.TestParallelImageWrite
            hadoop.hdfs.server.namenode.TestCheckpoint
            hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant
            hadoop.hdfs.server.namenode.snapshot.TestGetContentSummaryWithSnapshot
            hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys
            hadoop.hdfs.client.impl.TestLeaseRenewer
            hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
            hadoop.cli.TestCacheAdminCLI
            hadoop.hdfs.server.namenode.ha.TestRemoteNameNodeInfo
            hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot
            hadoop.hdfs.TestDatanodeDeath
            hadoop.hdfs.server.namenode.snapshot.TestSnapshot
            hadoop.hdfs.TestPipelines
            hadoop.hdfs.web.TestWebHDFSAcl
            hadoop.hdfs.server.namenode.TestAddBlockRetry
            hadoop.hdfs.server.namenode.TestFSImage
            hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
            hadoop.fs.TestUnbuffer
            hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes
            hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir
            hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional
            hadoop.hdfs.TestRenameWhileOpen
            hadoop.hdfs.server.namenode.TestNameNodeXAttr
            hadoop.hdfs.web.TestHttpsFileSystem
            hadoop.hdfs.server.namenode.TestEditLogJournalFailures
            hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
            hadoop.hdfs.TestWriteBlockGetsBlockLengthHint
            hadoop.hdfs.server.namenode.TestFSEditLogLoader
            hadoop.hdfs.tools.TestStoragePolicyCommands
            hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.TestDFSStartupVersions
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl
            hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
            hadoop.hdfs.TestAppendSnapshotTruncate
            hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.web.TestWebHdfsContentLength
            hadoop.hdfs.TestParallelShortCircuitReadUnCached
            hadoop.hdfs.TestDFSClientFailover
            hadoop.hdfs.TestFSOutputSummer
            hadoop.hdfs.server.namenode.TestHDFSConcat
            hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
            hadoop.hdfs.server.namenode.TestFSImageWithAcl
            hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
            hadoop.hdfs.web.TestFSMainOperationsWebHdfs
            hadoop.hdfs.TestDFSInotifyEventInputStream
            hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
            hadoop.hdfs.server.namenode.TestSaveNamespace
            hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
            hadoop.hdfs.TestBalancerBandwidth
            hadoop.hdfs.TestDFSPermission
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean
            hadoop.fs.viewfs.TestViewFileSystemHdfs
            hadoop.hdfs.TestListFilesInDFS
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.TestClusterId
            hadoop.hdfs.TestModTime
            hadoop.hdfs.TestReservedRawPaths
            hadoop.hdfs.server.namenode.TestFsLimits
            hadoop.hdfs.TestHDFSFileSystemContract
            hadoop.fs.contract.hdfs.TestHDFSContractSetTimes
            hadoop.hdfs.server.namenode.TestHostsFiles
            hadoop.hdfs.TestFileCreation
            hadoop.fs.viewfs.TestViewFsFileStatusHdfs
            hadoop.hdfs.TestDFSShellGenericOptions
            hadoop.hdfs.server.namenode.TestINodeAttributeProvider
            hadoop.hdfs.TestParallelShortCircuitRead
            hadoop.hdfs.server.namenode.TestFSImageWithXAttr
            hadoop.hdfs.TestHDFSTrash
            hadoop.hdfs.TestParallelShortCircuitLegacyRead
            hadoop.hdfs.TestFileCreationEmpty
            hadoop.hdfs.TestRead
            hadoop.hdfs.web.TestByteRangeInputStream
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.web.TestWebHdfsFileSystemContract
            hadoop.hdfs.server.namenode.TestSecureNameNode
            hadoop.hdfs.TestDFSAddressConfig
            hadoop.hdfs.TestDefaultNameNodePort
            hadoop.hdfs.web.TestWebHDFS
            hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
            hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
            hadoop.hdfs.server.namenode.TestAddBlock
            hadoop.tools.TestJMXGet
            hadoop.fs.TestFcHdfsCreateMkdir
            hadoop.fs.TestFcHdfsPermission
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.TestFetchImage
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.TestInjectionForSimulatedStorage
            hadoop.hdfs.server.namenode.TestValidateConfigurationSettings
            hadoop.hdfs.server.namenode.TestNameNodeMetricsLogger
            hadoop.hdfs.TestFileCreationClient
            hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure
            hadoop.hdfs.server.namenode.TestNameEditsConfigs
            hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.TestClientProtocolForPipelineRecovery
            hadoop.hdfs.web.TestJsonUtil
            hadoop.hdfs.TestDFSRemove
            hadoop.hdfs.TestFileAppendRestart
            hadoop.fs.contract.hdfs.TestHDFSContractCreate
            hadoop.net.TestNetworkTopology
            hadoop.hdfs.TestDFSOutputStream
            hadoop.hdfs.TestDisableConnCache
            hadoop.hdfs.TestParallelShortCircuitReadNoChecksum
            hadoop.hdfs.tools.TestGetGroups
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
            hadoop.hdfs.server.namenode.TestCreateEditsLog
            hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot
            hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
            hadoop.hdfs.server.namenode.TestStartup
            hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
            hadoop.hdfs.TestBlockMissingException
            hadoop.hdfs.protocolPB.TestPBHelper
            hadoop.hdfs.server.namenode.TestMalformedURLs
            hadoop.fs.permission.TestStickyBit
            hadoop.hdfs.server.namenode.TestFSPermissionChecker
            hadoop.hdfs.server.namenode.TestProtectedDirectories
          Timed out tests org.apache.hadoop.hdfs.server.namenode.TestEditLog



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12761458/HADOOP-12415.patch
          Optional Tests javadoc javac unit
          git revision trunk / c9cb6a5
          hadoop-nfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7683/artifact/patchprocess/testrun_hadoop-nfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7683/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7683/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7683/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 49s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 9m 5s There were no new javac warning messages. +1 javadoc 13m 48s There were no new javadoc warning messages. +1 release audit 0m 30s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 2m 5s mvn install still works. +1 eclipse:eclipse 0m 44s The patch built with eclipse:eclipse. +1 common tests 0m 27s Tests passed in hadoop-nfs. -1 hdfs tests 83m 39s Tests failed in hadoop-hdfs.     128m 12s   Reason Tests Failed unit tests hadoop.hdfs.TestWriteRead   hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots   hadoop.hdfs.server.namenode.ha.TestDNFencing   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.hdfs.TestHFlush   hadoop.security.TestPermission   hadoop.hdfs.TestParallelRead   hadoop.fs.viewfs.TestViewFsHdfs   hadoop.hdfs.TestBlockReaderLocalLegacy   hadoop.fs.TestHdfsNativeCodeLoader   hadoop.hdfs.server.namenode.TestAuditLogger   hadoop.hdfs.server.namenode.TestXAttrConfigFlag   hadoop.hdfs.TestPread   hadoop.hdfs.TestMiniDFSCluster   hadoop.hdfs.server.namenode.TestTransferFsImage   hadoop.hdfs.TestWriteConfigurationToDFS   hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.web.TestWebHDFSXAttr   hadoop.hdfs.TestDFSRollback   hadoop.hdfs.TestDataTransferKeepalive   hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA   hadoop.hdfs.TestDatanodeConfig   hadoop.hdfs.TestDFSFinalize   hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA   hadoop.fs.TestWebHdfsFileContextMainOperations   hadoop.hdfs.server.namenode.TestCommitBlockSynchronization   hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication   hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps   hadoop.fs.TestGlobPaths   hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.TestDFSShell   hadoop.hdfs.tools.TestDFSHAAdminMiniCluster   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.TestSeekBug   hadoop.fs.loadGenerator.TestLoadGenerator   hadoop.hdfs.TestCrcCorruption   hadoop.fs.contract.hdfs.TestHDFSContractMkdir   hadoop.hdfs.server.namenode.TestStorageRestore   hadoop.hdfs.TestAbandonBlock   hadoop.hdfs.TestGetFileChecksum   hadoop.hdfs.server.namenode.TestBlockUnderConstruction   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary   hadoop.hdfs.TestFileCreationDelete   hadoop.hdfs.TestReadWhileWriting   hadoop.fs.viewfs.TestViewFileSystemWithAcls   hadoop.fs.contract.hdfs.TestHDFSContractConcat   hadoop.hdfs.util.TestDiff   hadoop.hdfs.security.TestDelegationToken   hadoop.fs.TestSymlinkHdfsDisable   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory   hadoop.hdfs.TestMissingBlocksAlert   hadoop.hdfs.TestBlocksScheduledCounter   hadoop.hdfs.TestSmallBlock   hadoop.cli.TestDeleteCLI   hadoop.hdfs.TestDFSClientRetries   hadoop.fs.viewfs.TestViewFsWithXAttrs   hadoop.hdfs.TestDFSMkdirs   hadoop.hdfs.tools.TestDFSAdmin   hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.server.namenode.TestNameNodeMXBean   hadoop.hdfs.web.TestWebHDFSForHA   hadoop.hdfs.tools.TestGetConf   hadoop.hdfs.server.namenode.TestLargeDirectoryDelete   hadoop.fs.viewfs.TestViewFsDefaultValue   hadoop.fs.contract.hdfs.TestHDFSContractOpen   hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.TestFSInputChecker   hadoop.hdfs.web.TestWebHdfsWithAuthenticationFilter   hadoop.fs.contract.hdfs.TestHDFSContractRename   hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA   hadoop.hdfs.server.namenode.ha.TestHAMetrics   hadoop.hdfs.server.namenode.ha.TestLossyRetryInvocationHandler   hadoop.hdfs.TestRollingUpgradeRollback   hadoop.hdfs.TestRemoteBlockReader   hadoop.hdfs.server.namenode.TestNameNodeResourceChecker   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestLeaseRecovery   hadoop.fs.viewfs.TestViewFsAtHdfsRoot   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus   hadoop.cli.TestCryptoAdminCLI   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForXAttr   hadoop.hdfs.tools.TestDebugAdmin   hadoop.security.TestRefreshUserMappings   hadoop.hdfs.TestLargeBlock   hadoop.hdfs.server.namenode.TestFSNamesystemMBean   hadoop.fs.viewfs.TestViewFileSystemWithXAttrs   hadoop.hdfs.tools.TestDFSZKFailoverController   hadoop.hdfs.server.namenode.ha.TestNNHealthCheck   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.TestListFilesInFileContext   hadoop.hdfs.tools.TestDelegationTokenFetcher   hadoop.fs.TestFcHdfsSetUMask   hadoop.hdfs.server.namenode.TestNameNodeRecovery   hadoop.hdfs.TestDatanodeReport   hadoop.hdfs.TestFileAppend2   hadoop.fs.shell.TestHdfsTextCommand   hadoop.hdfs.TestFsShellPermission   hadoop.hdfs.TestHDFSServerPorts   hadoop.hdfs.server.namenode.ha.TestHAFsck   hadoop.hdfs.TestRollingUpgradeDowngrade   hadoop.hdfs.TestFileLengthOnClusterRestart   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.namenode.snapshot.TestSnapshotListing   hadoop.TestGenericRefresh   hadoop.hdfs.TestParallelUnixDomainRead   hadoop.hdfs.TestDFSUtil   hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication   hadoop.fs.TestSymlinkHdfsFileSystem   hadoop.hdfs.TestGetBlocks   hadoop.hdfs.security.token.block.TestBlockToken   hadoop.fs.contract.hdfs.TestHDFSContractAppend   hadoop.fs.contract.hdfs.TestHDFSContractDelete   hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot   hadoop.hdfs.TestClientReportBadBlock   hadoop.hdfs.TestDFSStorageStateRecovery   hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.cli.TestHDFSCLI   hadoop.fs.TestSWebHdfsFileContextMainOperations   hadoop.hdfs.TestRestartDFS   hadoop.hdfs.TestFileAppend4   hadoop.hdfs.web.TestWebHdfsUrl   hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters   hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover   hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled   hadoop.hdfs.TestSetTimes   hadoop.hdfs.server.namenode.TestAclConfigFlag   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.namenode.TestNameNodeAcl   hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy   hadoop.hdfs.server.namenode.TestSecondaryWebUi   hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.fs.contract.hdfs.TestHDFSContractSeek   hadoop.hdfs.TestSetrepIncreasing   hadoop.fs.viewfs.TestViewFsWithAcls   hadoop.hdfs.TestLease   hadoop.hdfs.TestSafeMode   hadoop.hdfs.server.namenode.TestEditLogAutoroll   hadoop.hdfs.TestDFSUpgrade   hadoop.hdfs.TestDecommission   hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer   hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   hadoop.fs.TestHDFSFileContextMainOperations   hadoop.hdfs.TestFileCorruption   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs   hadoop.hdfs.TestQuota   hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA   hadoop.hdfs.TestFileStatus   hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename   hadoop.hdfs.server.namenode.ha.TestHAConfiguration   hadoop.fs.TestResolveHdfsSymlink   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.hdfs.TestEncryptionZonesWithHA   hadoop.hdfs.server.namenode.TestDeadDatanode   hadoop.cli.TestXAttrCLI   hadoop.hdfs.server.namenode.TestProcessCorruptBlocks   hadoop.fs.TestUrlStreamHandler   hadoop.hdfs.TestClose   hadoop.hdfs.TestMultiThreadedHflush   hadoop.hdfs.TestDFSClientExcludedNodes   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport   hadoop.hdfs.server.namenode.TestEditLogRace   hadoop.hdfs.TestFileConcurrentReader   hadoop.hdfs.TestClientBlockVerification   hadoop.security.TestPermissionSymlinks   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.cli.TestAclCLI   hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots   hadoop.hdfs.TestDataTransferProtocol   hadoop.hdfs.server.namenode.TestParallelImageWrite   hadoop.hdfs.server.namenode.TestCheckpoint   hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant   hadoop.hdfs.server.namenode.snapshot.TestGetContentSummaryWithSnapshot   hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys   hadoop.hdfs.client.impl.TestLeaseRenewer   hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality   hadoop.cli.TestCacheAdminCLI   hadoop.hdfs.server.namenode.ha.TestRemoteNameNodeInfo   hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot   hadoop.hdfs.TestDatanodeDeath   hadoop.hdfs.server.namenode.snapshot.TestSnapshot   hadoop.hdfs.TestPipelines   hadoop.hdfs.web.TestWebHDFSAcl   hadoop.hdfs.server.namenode.TestAddBlockRetry   hadoop.hdfs.server.namenode.TestFSImage   hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes   hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   hadoop.fs.TestUnbuffer   hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes   hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir   hadoop.hdfs.server.namenode.TestNNStorageRetentionFunctional   hadoop.hdfs.TestRenameWhileOpen   hadoop.hdfs.server.namenode.TestNameNodeXAttr   hadoop.hdfs.web.TestHttpsFileSystem   hadoop.hdfs.server.namenode.TestEditLogJournalFailures   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   hadoop.hdfs.TestWriteBlockGetsBlockLengthHint   hadoop.hdfs.server.namenode.TestFSEditLogLoader   hadoop.hdfs.tools.TestStoragePolicyCommands   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.TestDFSStartupVersions   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl   hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   hadoop.hdfs.TestAppendSnapshotTruncate   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.web.TestWebHdfsContentLength   hadoop.hdfs.TestParallelShortCircuitReadUnCached   hadoop.hdfs.TestDFSClientFailover   hadoop.hdfs.TestFSOutputSummer   hadoop.hdfs.server.namenode.TestHDFSConcat   hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   hadoop.hdfs.server.namenode.TestFSImageWithAcl   hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot   hadoop.hdfs.web.TestFSMainOperationsWebHdfs   hadoop.hdfs.TestDFSInotifyEventInputStream   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement   hadoop.hdfs.server.namenode.TestSaveNamespace   hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics   hadoop.hdfs.TestBalancerBandwidth   hadoop.hdfs.TestDFSPermission   hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean   hadoop.fs.viewfs.TestViewFileSystemHdfs   hadoop.hdfs.TestListFilesInDFS   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.TestClusterId   hadoop.hdfs.TestModTime   hadoop.hdfs.TestReservedRawPaths   hadoop.hdfs.server.namenode.TestFsLimits   hadoop.hdfs.TestHDFSFileSystemContract   hadoop.fs.contract.hdfs.TestHDFSContractSetTimes   hadoop.hdfs.server.namenode.TestHostsFiles   hadoop.hdfs.TestFileCreation   hadoop.fs.viewfs.TestViewFsFileStatusHdfs   hadoop.hdfs.TestDFSShellGenericOptions   hadoop.hdfs.server.namenode.TestINodeAttributeProvider   hadoop.hdfs.TestParallelShortCircuitRead   hadoop.hdfs.server.namenode.TestFSImageWithXAttr   hadoop.hdfs.TestHDFSTrash   hadoop.hdfs.TestParallelShortCircuitLegacyRead   hadoop.hdfs.TestFileCreationEmpty   hadoop.hdfs.TestRead   hadoop.hdfs.web.TestByteRangeInputStream   hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics   hadoop.hdfs.TestFileAppend   hadoop.hdfs.web.TestWebHdfsFileSystemContract   hadoop.hdfs.server.namenode.TestSecureNameNode   hadoop.hdfs.TestDFSAddressConfig   hadoop.hdfs.TestDefaultNameNodePort   hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot   hadoop.hdfs.server.namenode.TestAddBlock   hadoop.tools.TestJMXGet   hadoop.fs.TestFcHdfsCreateMkdir   hadoop.fs.TestFcHdfsPermission   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.TestFetchImage   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.TestInjectionForSimulatedStorage   hadoop.hdfs.server.namenode.TestValidateConfigurationSettings   hadoop.hdfs.server.namenode.TestNameNodeMetricsLogger   hadoop.hdfs.TestFileCreationClient   hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure   hadoop.hdfs.server.namenode.TestNameEditsConfigs   hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.web.TestJsonUtil   hadoop.hdfs.TestDFSRemove   hadoop.hdfs.TestFileAppendRestart   hadoop.fs.contract.hdfs.TestHDFSContractCreate   hadoop.net.TestNetworkTopology   hadoop.hdfs.TestDFSOutputStream   hadoop.hdfs.TestDisableConnCache   hadoop.hdfs.TestParallelShortCircuitReadNoChecksum   hadoop.hdfs.tools.TestGetGroups   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer   hadoop.hdfs.server.namenode.TestCreateEditsLog   hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot   hadoop.hdfs.server.namenode.TestSecurityTokenEditLog   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot   hadoop.hdfs.TestBlockMissingException   hadoop.hdfs.protocolPB.TestPBHelper   hadoop.hdfs.server.namenode.TestMalformedURLs   hadoop.fs.permission.TestStickyBit   hadoop.hdfs.server.namenode.TestFSPermissionChecker   hadoop.hdfs.server.namenode.TestProtectedDirectories Timed out tests org.apache.hadoop.hdfs.server.namenode.TestEditLog Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761458/HADOOP-12415.patch Optional Tests javadoc javac unit git revision trunk / c9cb6a5 hadoop-nfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7683/artifact/patchprocess/testrun_hadoop-nfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7683/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7683/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7683/console This message was automatically generated.
          Hide
          tomzeng Tom Zeng added a comment -

          Steve Loughran I checked the hdfs tests error log, the class not found exceptions do not seem to be related to the patch, any way you can verify tests pass without the patch? thanks.

          Show
          tomzeng Tom Zeng added a comment - Steve Loughran I checked the hdfs tests error log, the class not found exceptions do not seem to be related to the patch, any way you can verify tests pass without the patch? thanks.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          resubmitting. I'm pretty happy with the patch though; just doing this for rigorousness. Its not adding more dependencies to modules that already depend on it, after all

          Show
          stevel@apache.org Steve Loughran added a comment - resubmitting. I'm pretty happy with the patch though; just doing this for rigorousness. Its not adding more dependencies to modules that already depend on it, after all
          Hide
          cos Konstantin Boudnik added a comment -

          Let's get it in, then?

          Show
          cos Konstantin Boudnik added a comment - Let's get it in, then?
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1 from me. Kos -do you want to do the commit?

          Show
          stevel@apache.org Steve Loughran added a comment - +1 from me. Kos -do you want to do the commit?
          Hide
          cos Konstantin Boudnik added a comment -

          I will commit this later in the day, Steve. Thanks for the review.

          Show
          cos Konstantin Boudnik added a comment - I will commit this later in the day, Steve. Thanks for the review.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 6s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 3m 17s trunk passed
          +1 compile 4m 21s trunk passed with JDK v1.8.0_60
          +1 compile 4m 8s trunk passed with JDK v1.7.0_79
          +1 mvneclipse 0m 26s trunk passed
          +1 javadoc 1m 20s trunk passed with JDK v1.8.0_60
          +1 javadoc 2m 1s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 53s the patch passed
          +1 compile 4m 17s the patch passed with JDK v1.8.0_60
          +1 javac 4m 17s the patch passed
          +1 compile 4m 7s the patch passed with JDK v1.7.0_79
          +1 javac 4m 8s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 1m 18s the patch passed with JDK v1.8.0_60
          +1 javadoc 2m 4s the patch passed with JDK v1.7.0_79
          +1 unit 0m 19s hadoop-nfs in the patch passed with JDK v1.8.0_60.
          -1 unit 50m 29s hadoop-hdfs in the patch failed with JDK v1.8.0_60.
          +1 unit 0m 23s hadoop-nfs in the patch passed with JDK v1.7.0_79.
          -1 unit 49m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 22s Patch generated 56 ASF License warnings.
          131m 55s



          Reason Tests
          JDK v1.8.0_60 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount
          JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-06
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12761458/HADOOP-12415.patch
          JIRA Issue HADOOP-12415
          Optional Tests asflicense javac javadoc mvninstall unit xml compile
          uname Linux 5ac3fa9e8e69 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / 286cc64
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-nfs hadoop-hdfs-project/hadoop-hdfs U: .
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 6s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 3m 17s trunk passed +1 compile 4m 21s trunk passed with JDK v1.8.0_60 +1 compile 4m 8s trunk passed with JDK v1.7.0_79 +1 mvneclipse 0m 26s trunk passed +1 javadoc 1m 20s trunk passed with JDK v1.8.0_60 +1 javadoc 2m 1s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 53s the patch passed +1 compile 4m 17s the patch passed with JDK v1.8.0_60 +1 javac 4m 17s the patch passed +1 compile 4m 7s the patch passed with JDK v1.7.0_79 +1 javac 4m 8s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 1m 18s the patch passed with JDK v1.8.0_60 +1 javadoc 2m 4s the patch passed with JDK v1.7.0_79 +1 unit 0m 19s hadoop-nfs in the patch passed with JDK v1.8.0_60. -1 unit 50m 29s hadoop-hdfs in the patch failed with JDK v1.8.0_60. +1 unit 0m 23s hadoop-nfs in the patch passed with JDK v1.7.0_79. -1 unit 49m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 22s Patch generated 56 ASF License warnings. 131m 55s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130 Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-06 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12761458/HADOOP-12415.patch JIRA Issue HADOOP-12415 Optional Tests asflicense javac javadoc mvninstall unit xml compile uname Linux 5ac3fa9e8e69 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 286cc64 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-nfs hadoop-hdfs-project/hadoop-hdfs U: . Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8036/console This message was automatically generated.
          Hide
          tomzeng Tom Zeng added a comment -

          Again the test failures do not seems to be related to the commit, and way to verify all tests pass without the patch?

          Show
          tomzeng Tom Zeng added a comment - Again the test failures do not seems to be related to the commit, and way to verify all tests pass without the patch?
          Hide
          cos Konstantin Boudnik added a comment -

          I think we're good here. Sorry, I dropped the ball - should've been committing this a while ago. Will do over the weekend.

          Show
          cos Konstantin Boudnik added a comment - I think we're good here. Sorry, I dropped the ball - should've been committing this a while ago. Will do over the weekend.
          Hide
          cos Konstantin Boudnik added a comment -

          Ok, I have pushed the fix into trunk and branch-2
          Looks like 2.7.2 is still going, can I add into the train? Will do in the morning unless hear otherwise. Thanks!

          Show
          cos Konstantin Boudnik added a comment - Ok, I have pushed the fix into trunk and branch-2 Looks like 2.7.2 is still going, can I add into the train? Will do in the morning unless hear otherwise. Thanks!
          Hide
          cos Konstantin Boudnik added a comment -

          The fix has been pushed. Thanks for the contribution, Tom!

          Show
          cos Konstantin Boudnik added a comment - The fix has been pushed. Thanks for the contribution, Tom!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8883 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8883/)
          HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f)

          • hadoop-common-project/hadoop-nfs/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8883 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8883/ ) HADOOP-12415 . Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f) hadoop-common-project/hadoop-nfs/pom.xml hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1452 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1452/)
          HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f)

          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • hadoop-common-project/hadoop-nfs/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1452 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1452/ ) HADOOP-12415 . Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f) hadoop-hdfs-project/hadoop-hdfs/pom.xml hadoop-common-project/hadoop-nfs/pom.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #720 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/720/)
          HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f)

          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • hadoop-common-project/hadoop-nfs/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #720 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/720/ ) HADOOP-12415 . Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f) hadoop-hdfs-project/hadoop-hdfs/pom.xml hadoop-common-project/hadoop-nfs/pom.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #730 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/730/)
          HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f)

          • hadoop-common-project/hadoop-nfs/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #730 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/730/ ) HADOOP-12415 . Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f) hadoop-common-project/hadoop-nfs/pom.xml hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2662 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2662/)
          HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f)

          • hadoop-common-project/hadoop-nfs/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2662 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2662/ ) HADOOP-12415 . Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f) hadoop-common-project/hadoop-nfs/pom.xml hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #641 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/641/)
          HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f)

          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • hadoop-common-project/hadoop-nfs/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #641 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/641/ ) HADOOP-12415 . Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f) hadoop-hdfs-project/hadoop-hdfs/pom.xml hadoop-common-project/hadoop-nfs/pom.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2578 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2578/)
          HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f)

          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • hadoop-common-project/hadoop-nfs/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2578 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2578/ ) HADOOP-12415 . Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, (cos: rev b4c6b511e705230dfe4580288addacb81f5c5c5f) hadoop-hdfs-project/hadoop-hdfs/pom.xml hadoop-common-project/hadoop-nfs/pom.xml
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Kos, target 2.7.3 rather than 2.7.2 if you can ... only critical issues should be going into 2.7.2 at this stage.

          Show
          stevel@apache.org Steve Loughran added a comment - Kos, target 2.7.3 rather than 2.7.2 if you can ... only critical issues should be going into 2.7.2 at this stage.
          Hide
          cos Konstantin Boudnik added a comment -

          Steve, arguably this is critical because the build is broken when ran from a clean slate. I don't mind one way or another, as we just patched the Hadoop in the Bigtop stack, but ppl using it directly from the release artifacts would be affected.

          Show
          cos Konstantin Boudnik added a comment - Steve, arguably this is critical because the build is broken when ran from a clean slate. I don't mind one way or another, as we just patched the Hadoop in the Bigtop stack, but ppl using it directly from the release artifacts would be affected.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          check with vinod then

          Show
          stevel@apache.org Steve Loughran added a comment - check with vinod then
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          How can we reproduce this from our end? I am surprised that HADOOP-11489 sneaked through without any of us catching through the numerous builds that happened during the release, jenkins etc.

          The patch seems to not to be on point though - HADOOP-11489 only removed the dependency from hadoop-nfs and this patch is adding it all the way up into hadoop-common and hadoop-hdfs. Shouldn't the patch be simply a revert of HADOOP-11489?

          Overall, we can include the right fix for this JIRA in 2.7.2 as I am still stuck on a couple of HDFS patches.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - How can we reproduce this from our end? I am surprised that HADOOP-11489 sneaked through without any of us catching through the numerous builds that happened during the release, jenkins etc. The patch seems to not to be on point though - HADOOP-11489 only removed the dependency from hadoop-nfs and this patch is adding it all the way up into hadoop-common and hadoop-hdfs. Shouldn't the patch be simply a revert of HADOOP-11489 ? Overall, we can include the right fix for this JIRA in 2.7.2 as I am still stuck on a couple of HDFS patches.
          Hide
          cos Konstantin Boudnik added a comment -

          The way to reproduce it is described in the JIRA - just look through the comments, it should be close to the top.
          The reason we didn't simply reverted HADOOP-11489 is because the current fix presents a cleaner solution. Besides, Steve asked to make it more fine-grained in here

          Show
          cos Konstantin Boudnik added a comment - The way to reproduce it is described in the JIRA - just look through the comments, it should be close to the top. The reason we didn't simply reverted HADOOP-11489 is because the current fix presents a cleaner solution. Besides, Steve asked to make it more fine-grained in here
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Vinod: it happens because some changes upstream haven't been picked up in Hadoop's dependencies: it works with the stock dependency set, but not when we've moved on. So it will surface.

          I proposed keeping it out of reverting the change as it was part of the ongoing split to isolate client and server side code. These patches make sure netty is in there server-side

          If you're happy with it going in to 2.7.2, then Konstantin; +1 from me

          Show
          stevel@apache.org Steve Loughran added a comment - Vinod: it happens because some changes upstream haven't been picked up in Hadoop's dependencies: it works with the stock dependency set, but not when we've moved on. So it will surface . I proposed keeping it out of reverting the change as it was part of the ongoing split to isolate client and server side code. These patches make sure netty is in there server-side If you're happy with it going in to 2.7.2, then Konstantin; +1 from me
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Merged this into branch-2.7, branch-2.7.2 per our discussion above.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Merged this into branch-2.7, branch-2.7.2 per our discussion above.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8979 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8979/)
          HADOOP-12415. Fixed pom files to correctly include compile-time (vinodkv: rev ce16541c62728f69634111dc2ddb321690d3d29b)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8979 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8979/ ) HADOOP-12415 . Fixed pom files to correctly include compile-time (vinodkv: rev ce16541c62728f69634111dc2ddb321690d3d29b) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          cos Konstantin Boudnik added a comment -

          Thanks Vinod!

          Show
          cos Konstantin Boudnik added a comment - Thanks Vinod!

            People

            • Assignee:
              tomzeng Tom Zeng
              Reporter:
              cos Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development