Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9574

Reduce client failures during datanode restart

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.2, 2.6.4, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Since DataXceiverServer is initialized before BP is fully up, client requests will fail until the datanode registers.

      1. HDFS-9574.patch
        15 kB
        Kihwal Lee
      2. HDFS-9574.v2.patch
        17 kB
        Kihwal Lee
      3. HDFS-9574.v3.br26.patch
        27 kB
        Kihwal Lee
      4. HDFS-9574.v3.br27.patch
        26 kB
        Kihwal Lee
      5. HDFS-9574.v3.patch
        28 kB
        Kihwal Lee

        Issue Links

          Activity

          Hide
          kihwal Kihwal Lee added a comment -

          Read fails like this:

          java.io.IOException: cannot find BPOfferService for bpid=BP-xxx
                  at org.apache.hadoop.hdfs.server.datanode.DataNode.getDNRegistrationForBP(DataNode.java:1434)
                  at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:525)
                  at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:116)
                  at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:71)
                  at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:251)
                  at java.lang.Thread.run(Thread.java:745)
          

          Writes fail like this:

          java.lang.NullPointerException
                  at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.<init>(BlockReceiver.java:200)
                  at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:679)
                  at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:137)
                  at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:74)
                  at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:251)
                  at java.lang.Thread.run(Thread.java:745)
          
          Show
          kihwal Kihwal Lee added a comment - Read fails like this: java.io.IOException: cannot find BPOfferService for bpid=BP-xxx at org.apache.hadoop.hdfs.server.datanode.DataNode.getDNRegistrationForBP(DataNode.java:1434) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:525) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:116) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:71) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:251) at java.lang.Thread.run(Thread.java:745) Writes fail like this: java.lang.NullPointerException at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.<init>(BlockReceiver.java:200) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:679) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:137) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:74) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:251) at java.lang.Thread.run(Thread.java:745)
          Hide
          kihwal Kihwal Lee added a comment -

          Since DataXceiverServer is shared by multiple block pools, we cannot simply move its initialization to later point. I will have each data transfer op wait up to configured amount of time before giving up. A number of threads will pile up until the node registers, but it cannot serve anyway, so I don't think we are introducing any more service degradation.

          Show
          kihwal Kihwal Lee added a comment - Since DataXceiverServer is shared by multiple block pools, we cannot simply move its initialization to later point. I will have each data transfer op wait up to configured amount of time before giving up. A number of threads will pile up until the node registers, but it cannot serve anyway, so I don't think we are introducing any more service degradation.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 54s trunk passed
          +1 compile 1m 6s trunk passed with JDK v1.8.0_66
          +1 compile 0m 53s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 1m 9s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 2m 29s trunk passed
          +1 javadoc 1m 35s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 24s trunk passed with JDK v1.7.0_91
          +1 mvninstall 1m 4s the patch passed
          +1 compile 1m 10s the patch passed with JDK v1.8.0_66
          +1 javac 1m 10s the patch passed
          +1 compile 0m 56s the patch passed with JDK v1.7.0_91
          +1 javac 0m 56s the patch passed
          -1 checkstyle 0m 21s Patch generated 4 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 612, now 613).
          +1 mvnsite 1m 8s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 35s the patch passed
          +1 javadoc 1m 29s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 29s the patch passed with JDK v1.7.0_91
          -1 unit 78m 26s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 71m 14s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          -1 asflicense 0m 30s Patch generated 56 ASF License warnings.
          185m 6s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.TestDatanodeRegistration
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
            hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint
            hadoop.hdfs.TestRecoverStripedFile
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778376/HDFS-9574.patch
          JIRA Issue HDFS-9574
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 400ccc3fcbcf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1de56b0
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13926/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13926/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 54s trunk passed +1 compile 1m 6s trunk passed with JDK v1.8.0_66 +1 compile 0m 53s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 22s trunk passed +1 mvnsite 1m 9s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 2m 29s trunk passed +1 javadoc 1m 35s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 24s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 4s the patch passed +1 compile 1m 10s the patch passed with JDK v1.8.0_66 +1 javac 1m 10s the patch passed +1 compile 0m 56s the patch passed with JDK v1.7.0_91 +1 javac 0m 56s the patch passed -1 checkstyle 0m 21s Patch generated 4 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 612, now 613). +1 mvnsite 1m 8s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 35s the patch passed +1 javadoc 1m 29s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 29s the patch passed with JDK v1.7.0_91 -1 unit 78m 26s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 71m 14s hadoop-hdfs in the patch failed with JDK v1.7.0_91. -1 asflicense 0m 30s Patch generated 56 ASF License warnings. 185m 6s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040   hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint   hadoop.hdfs.TestRecoverStripedFile JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030 Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778376/HDFS-9574.patch JIRA Issue HDFS-9574 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 400ccc3fcbcf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1de56b0 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13926/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13926/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13926/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          All failed test cases pass when run locally, except this one.

          Running org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint
          Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 48.968 sec <<< FAILURE!
           - in org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint
          testWithRemoteClient(org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint)  Time elapsed: 16.784 sec  <<< ERROR!
          java.io.IOException: Not ready to serve the block pool, Dummy-pool.
          	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.checkAndWaitForBP(DataXceiver.java:218)
          	at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.issueWriteBlockCall(TestDataXceiverLazyPersistHint.java:118)
          	at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.testWithRemoteClient(TestDataXceiverLazyPersistHint.java:90)
          
          testOverrideWithRemoteClient(org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint)  Time elapsed: 16.047 sec  <<< ERROR!
          java.io.IOException: Not ready to serve the block pool, Dummy-pool.
          	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.checkAndWaitForBP(DataXceiver.java:218)
          	at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.issueWriteBlockCall(TestDataXceiverLazyPersistHint.java:118)
          	at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.testOverrideWithRemoteClient(TestDataXceiverLazyPersistHint.java:107)
          
          testWithLocalClient(org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint)  Time elapsed: 16.049 sec  <<< ERROR!
          java.io.IOException: Not ready to serve the block pool, Dummy-pool.
          	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.checkAndWaitForBP(DataXceiver.java:218)
          	at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.issueWriteBlockCall(TestDataXceiverLazyPersistHint.java:118)
          	at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.testWithLocalClient(TestDataXceiverLazyPersistHint.java:75)
          
          Show
          kihwal Kihwal Lee added a comment - All failed test cases pass when run locally, except this one. Running org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 48.968 sec <<< FAILURE! - in org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint testWithRemoteClient(org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint) Time elapsed: 16.784 sec <<< ERROR! java.io.IOException: Not ready to serve the block pool, Dummy-pool. at org.apache.hadoop.hdfs.server.datanode.DataXceiver.checkAndWaitForBP(DataXceiver.java:218) at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.issueWriteBlockCall(TestDataXceiverLazyPersistHint.java:118) at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.testWithRemoteClient(TestDataXceiverLazyPersistHint.java:90) testOverrideWithRemoteClient(org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint) Time elapsed: 16.047 sec <<< ERROR! java.io.IOException: Not ready to serve the block pool, Dummy-pool. at org.apache.hadoop.hdfs.server.datanode.DataXceiver.checkAndWaitForBP(DataXceiver.java:218) at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.issueWriteBlockCall(TestDataXceiverLazyPersistHint.java:118) at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.testOverrideWithRemoteClient(TestDataXceiverLazyPersistHint.java:107) testWithLocalClient(org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint) Time elapsed: 16.049 sec <<< ERROR! java.io.IOException: Not ready to serve the block pool, Dummy-pool. at org.apache.hadoop.hdfs.server.datanode.DataXceiver.checkAndWaitForBP(DataXceiver.java:218) at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.issueWriteBlockCall(TestDataXceiverLazyPersistHint.java:118) at org.apache.hadoop.hdfs.server.datanode.TestDataXceiverLazyPersistHint.testWithLocalClient(TestDataXceiverLazyPersistHint.java:75)
          Hide
          kihwal Kihwal Lee added a comment -

          Fixed the broken test by stubbing the getDNRegistrationForBP() method for the existing mocked DataNode.

          Show
          kihwal Kihwal Lee added a comment - Fixed the broken test by stubbing the getDNRegistrationForBP() method for the existing mocked DataNode .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 18m 35s trunk passed
          +1 compile 2m 39s trunk passed with JDK v1.8.0_66
          +1 compile 2m 2s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 38s trunk passed
          +1 mvnsite 2m 26s trunk passed
          +1 mvneclipse 0m 33s trunk passed
          +1 findbugs 5m 12s trunk passed
          +1 javadoc 3m 9s trunk passed with JDK v1.8.0_66
          +1 javadoc 4m 48s trunk passed with JDK v1.7.0_91
          +1 mvninstall 2m 24s the patch passed
          +1 compile 3m 1s the patch passed with JDK v1.8.0_66
          +1 javac 3m 1s the patch passed
          +1 compile 2m 7s the patch passed with JDK v1.7.0_91
          +1 javac 2m 7s the patch passed
          -1 checkstyle 0m 43s Patch generated 3 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 612, now 612).
          +1 mvnsite 2m 30s the patch passed
          +1 mvneclipse 0m 33s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 5m 7s the patch passed
          +1 javadoc 2m 59s the patch passed with JDK v1.8.0_66
          +1 javadoc 4m 37s the patch passed with JDK v1.7.0_91
          -1 unit 214m 11s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 216m 6s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 1m 41s Patch does not generate ASF License warnings.
          504m 56s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
            hadoop.hdfs.server.namenode.TestFSEditLogLoader
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.TestDFSUpgrade
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.TestQuota
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.TestPersistBlocks
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
            hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.TestRecoverStripedFile
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.TestLocalDFS
            hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
            hadoop.hdfs.server.namenode.TestStripedINodeFile
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
            hadoop.hdfs.server.namenode.TestMetadataVersionOutput
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.hdfs.tools.TestDFSAdminWithHA
            hadoop.hdfs.web.TestWebHdfsTokens
            hadoop.hdfs.TestDFSStripedOutputStream
            hadoop.hdfs.TestDatanodeRegistration
            hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
            hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
            hadoop.fs.TestSymlinkHdfsFileContext
            hadoop.fs.TestSymlinkHdfsFileSystem
            hadoop.hdfs.qjournal.TestSecureNNWithQJM
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport
            hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
            hadoop.hdfs.server.namenode.TestDeleteRace
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.mover.TestMover
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.TestDFSUpgrade
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.server.namenode.TestSecureNameNode
            hadoop.hdfs.TestModTime
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180
            hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.TestDataTransferKeepalive
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.TestLocalDFS
            hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.server.namenode.TestMetadataVersionOutput
            hadoop.hdfs.TestDecommission
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020
            hadoop.hdfs.TestSafeModeWithStripedFile
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.hdfs.tools.TestDFSAdminWithHA
            hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
            hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation
            hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.TestDFSShell
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
            hadoop.hdfs.TestReplication
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.fs.TestSymlinkHdfsFileContext
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130
            hadoop.fs.TestSymlinkHdfsFileSystem
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.qjournal.TestSecureNNWithQJM
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          JDK v1.7.0_91 Timed out junit tests org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778500/HDFS-9574.v2.patch
          JIRA Issue HDFS-9574
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 4fb5ce89c92b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / bfadf11
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13934/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13934/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 18m 35s trunk passed +1 compile 2m 39s trunk passed with JDK v1.8.0_66 +1 compile 2m 2s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 38s trunk passed +1 mvnsite 2m 26s trunk passed +1 mvneclipse 0m 33s trunk passed +1 findbugs 5m 12s trunk passed +1 javadoc 3m 9s trunk passed with JDK v1.8.0_66 +1 javadoc 4m 48s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 24s the patch passed +1 compile 3m 1s the patch passed with JDK v1.8.0_66 +1 javac 3m 1s the patch passed +1 compile 2m 7s the patch passed with JDK v1.7.0_91 +1 javac 2m 7s the patch passed -1 checkstyle 0m 43s Patch generated 3 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 612, now 612). +1 mvnsite 2m 30s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 5m 7s the patch passed +1 javadoc 2m 59s the patch passed with JDK v1.8.0_66 +1 javadoc 4m 37s the patch passed with JDK v1.7.0_91 -1 unit 214m 11s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 216m 6s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 1m 41s Patch does not generate ASF License warnings. 504m 56s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.hdfs.server.namenode.TestFSEditLogLoader   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.TestDFSUpgrade   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.TestQuota   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.TestRecoverStripedFile   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.TestLocalDFS   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170   hadoop.hdfs.server.namenode.TestStripedINodeFile   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160   hadoop.hdfs.server.namenode.TestMetadataVersionOutput   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.TestDFSStripedOutputStream   hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.server.namenode.TestSecurityTokenEditLog   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.fs.TestSymlinkHdfsFileSystem   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.namenode.TestFSImageWithSnapshot   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport   hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork   hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.mover.TestMover   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.TestDFSUpgrade   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.namenode.TestSecureNameNode   hadoop.hdfs.TestModTime   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.TestDataTransferKeepalive   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.TestLocalDFS   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.server.namenode.TestMetadataVersionOutput   hadoop.hdfs.TestDecommission   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020   hadoop.hdfs.TestSafeModeWithStripedFile   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.namenode.TestSecurityTokenEditLog   hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.TestDFSShell   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.TestReplication   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130   hadoop.fs.TestSymlinkHdfsFileSystem   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_91 Timed out junit tests org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778500/HDFS-9574.v2.patch JIRA Issue HDFS-9574 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 4fb5ce89c92b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bfadf11 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13934/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13934/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13934/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          This precommit (#13934) and #13935 for HDFS-7163 ran on H9 at the same time, causing both to run for over 5 hours and failing many tests due to internal timeout.
          Allen Wittenauer, do you think we can configure jenkins to avoid this? Since HDFS tests are run in parallel, concurrent hdfs builds are worse than before.

          Show
          kihwal Kihwal Lee added a comment - This precommit (#13934) and #13935 for HDFS-7163 ran on H9 at the same time, causing both to run for over 5 hours and failing many tests due to internal timeout. Allen Wittenauer , do you think we can configure jenkins to avoid this? Since HDFS tests are run in parallel, concurrent hdfs builds are worse than before.
          Hide
          aw Allen Wittenauer added a comment -

          do you think we can configure jenkins to avoid this?

          No.

          Since HDFS tests are run in parallel, concurrent hdfs builds are worse than before.

          Maybe this will finally be the push folks need to clean up the tests.

          Show
          aw Allen Wittenauer added a comment - do you think we can configure jenkins to avoid this? No. Since HDFS tests are run in parallel, concurrent hdfs builds are worse than before. Maybe this will finally be the push folks need to clean up the tests.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 7m 50s trunk passed
          +1 compile 0m 41s trunk passed with JDK v1.8.0_66
          +1 compile 0m 41s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 16s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 57s trunk passed
          +1 javadoc 1m 7s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 49s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 50s the patch passed
          +1 compile 0m 42s the patch passed with JDK v1.8.0_66
          +1 javac 0m 42s the patch passed
          +1 compile 0m 42s the patch passed with JDK v1.7.0_91
          +1 javac 0m 42s the patch passed
          -1 checkstyle 0m 16s Patch generated 3 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 613, now 613).
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 17s the patch passed
          +1 javadoc 1m 19s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 1s the patch passed with JDK v1.7.0_91
          -1 unit 83m 30s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 74m 14s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 32s Patch does not generate ASF License warnings.
          186m 29s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.TestDatanodeRegistration
            hadoop.hdfs.server.namenode.TestBackupNode
            hadoop.hdfs.shortcircuit.TestShortCircuitCache
            hadoop.hdfs.server.datanode.TestBlockReplacement
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778500/HDFS-9574.v2.patch
          JIRA Issue HDFS-9574
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 668410dc6741 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a0de702
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13964/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13964/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 50s trunk passed +1 compile 0m 41s trunk passed with JDK v1.8.0_66 +1 compile 0m 41s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 57s trunk passed +1 javadoc 1m 7s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 49s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 50s the patch passed +1 compile 0m 42s the patch passed with JDK v1.8.0_66 +1 javac 0m 42s the patch passed +1 compile 0m 42s the patch passed with JDK v1.7.0_91 +1 javac 0m 42s the patch passed -1 checkstyle 0m 16s Patch generated 3 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 613, now 613). +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 17s the patch passed +1 javadoc 1m 19s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 1s the patch passed with JDK v1.7.0_91 -1 unit 83m 30s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 74m 14s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 32s Patch does not generate ASF License warnings. 186m 29s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.server.datanode.TestBlockReplacement JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778500/HDFS-9574.v2.patch JIRA Issue HDFS-9574 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 668410dc6741 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a0de702 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13964/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13964/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13964/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -
          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement
          Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.643 sec - in org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.datanode.TestBlockScanner
          Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.741 sec - in org.apache.hadoop.hdfs.server.datanode.TestBlockScanner
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.namenode.TestBackupNode
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.891 sec - in org.apache.hadoop.hdfs.server.namenode.TestBackupNode
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.523 sec - in org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache
          Tests run: 11, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 1.102 sec - in org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.TestDatanodeRegistration
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.489 sec - in org.apache.hadoop.hdfs.TestDatanodeRegistration
          
          Results :
          
          Tests run: 39, Failures: 0, Errors: 0, Skipped: 6
          
          Show
          kihwal Kihwal Lee added a comment - ------------------------------------------------------- T E S T S ------------------------------------------------------- Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 29.643 sec - in org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.datanode.TestBlockScanner Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.741 sec - in org.apache.hadoop.hdfs.server.datanode.TestBlockScanner Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.namenode.TestBackupNode Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.891 sec - in org.apache.hadoop.hdfs.server.namenode.TestBackupNode Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.523 sec - in org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache Tests run: 11, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 1.102 sec - in org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitCache Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.TestDatanodeRegistration Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.489 sec - in org.apache.hadoop.hdfs.TestDatanodeRegistration Results : Tests run: 39, Failures: 0, Errors: 0, Skipped: 6
          Hide
          kihwal Kihwal Lee added a comment -
           mvn test -Dtest=TestDFSStripedOutputStreamWithFailure*
          ...
          Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110
          Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.532 sec - in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110
          ...
          Results :
          Tests run: 224, Failures: 0, Errors: 0, Skipped: 0
          

          All reported test failures are not reproduced when run locally.

          Show
          kihwal Kihwal Lee added a comment - mvn test -Dtest=TestDFSStripedOutputStreamWithFailure* ... Running org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110 Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.532 sec - in org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110 ... Results : Tests run: 224, Failures: 0, Errors: 0, Skipped: 0 All reported test failures are not reproduced when run locally.
          Hide
          daryn Daryn Sharp added a comment -

          Might consider checking if the bp is registered in checkAccess to avoid every caller explicitly checking the bp before calling checkAccess.

          Sleeping for 1s and and incrementing a counter until it reaches the number of configured seconds is fragile - assumes the sleep really was for 1s which may not be true if there was a long GC, etc. I'd suggest using a StopWatch for correctness.

          I think something similar needs to be done for the RPC service. Block tokens cannot be authenticated until after registration when it gets the block secret. The dfs client checks getReplicaVisibleLength for the last block if not complete and the rpc client doesn't appear to have any retry proxy. This is likely to affect users that frequently read while writing or appending to a file (ex. logging into hdfs, perhaps hbase?).

          Blocking in the RPC layer, unlike the data xceiver threads, is not desirable. Once the readers jam due to one unregistered bp, admin calls or calls for other block pools will be stalled too. Ideally the DN secret manager should throw a RetriableException if the bp has no secrets. The client can handle the retries. Appears it would be backwards compat.

          Show
          daryn Daryn Sharp added a comment - Might consider checking if the bp is registered in checkAccess to avoid every caller explicitly checking the bp before calling checkAccess . Sleeping for 1s and and incrementing a counter until it reaches the number of configured seconds is fragile - assumes the sleep really was for 1s which may not be true if there was a long GC, etc. I'd suggest using a StopWatch for correctness. I think something similar needs to be done for the RPC service. Block tokens cannot be authenticated until after registration when it gets the block secret. The dfs client checks getReplicaVisibleLength for the last block if not complete and the rpc client doesn't appear to have any retry proxy. This is likely to affect users that frequently read while writing or appending to a file (ex. logging into hdfs, perhaps hbase?). Blocking in the RPC layer, unlike the data xceiver threads, is not desirable. Once the readers jam due to one unregistered bp, admin calls or calls for other block pools will be stalled too. Ideally the DN secret manager should throw a RetriableException if the bp has no secrets. The client can handle the retries. Appears it would be backwards compat.
          Hide
          kihwal Kihwal Lee added a comment -

          The new patch addresses the review comments.

          • All relevant DataTransfer methods are now calling checkAccess() and the registration is checked from there.
          • The elapsed time is now tracked using StopWatch.
          • getReplicaVisibleLength() now throws RetriableException.
          • DFSInpuStream retries those nodes that threw RetriableException on getReplicaVisibleLength(), with a limit. The client read timeout is used for the retry timeout.
          • The test case was expanded to cover the getReplicaVisibleLength() case.
          Show
          kihwal Kihwal Lee added a comment - The new patch addresses the review comments. All relevant DataTransfer methods are now calling checkAccess() and the registration is checked from there. The elapsed time is now tracked using StopWatch . getReplicaVisibleLength() now throws RetriableException . DFSInpuStream retries those nodes that threw RetriableException on getReplicaVisibleLength() , with a limit. The client read timeout is used for the retry timeout. The test case was expanded to cover the getReplicaVisibleLength() case.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 8m 39s trunk passed
          +1 compile 1m 50s trunk passed with JDK v1.8.0_66
          +1 compile 1m 50s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 34s trunk passed
          +1 mvnsite 1m 40s trunk passed
          +1 mvneclipse 0m 31s trunk passed
          +1 findbugs 4m 13s trunk passed
          +1 javadoc 1m 46s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 29s trunk passed with JDK v1.7.0_91
          +1 mvninstall 1m 23s the patch passed
          +1 compile 1m 38s the patch passed with JDK v1.8.0_66
          +1 javac 1m 38s the patch passed
          +1 compile 1m 42s the patch passed with JDK v1.7.0_91
          +1 javac 1m 43s the patch passed
          -1 checkstyle 0m 33s Patch generated 3 new checkstyle issues in hadoop-hdfs-project (total was 692, now 690).
          +1 mvnsite 1m 33s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 4m 31s the patch passed
          +1 javadoc 1m 36s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 27s the patch passed with JDK v1.7.0_91
          +1 unit 0m 53s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
          -1 unit 71m 5s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91.
          -1 unit 68m 6s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 26s Patch does not generate ASF License warnings.
          183m 57s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.TestFsck
            hadoop.hdfs.TestSafeMode



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781107/HDFS-9574.v3.patch
          JIRA Issue HDFS-9574
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 09133d95bfa6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 89022f8
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14062/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project
          Max memory used 75MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14062/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 39s trunk passed +1 compile 1m 50s trunk passed with JDK v1.8.0_66 +1 compile 1m 50s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 34s trunk passed +1 mvnsite 1m 40s trunk passed +1 mvneclipse 0m 31s trunk passed +1 findbugs 4m 13s trunk passed +1 javadoc 1m 46s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 29s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 23s the patch passed +1 compile 1m 38s the patch passed with JDK v1.8.0_66 +1 javac 1m 38s the patch passed +1 compile 1m 42s the patch passed with JDK v1.7.0_91 +1 javac 1m 43s the patch passed -1 checkstyle 0m 33s Patch generated 3 new checkstyle issues in hadoop-hdfs-project (total was 692, now 690). +1 mvnsite 1m 33s the patch passed +1 mvneclipse 0m 25s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 4m 31s the patch passed +1 javadoc 1m 36s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 27s the patch passed with JDK v1.7.0_91 +1 unit 0m 53s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. -1 unit 71m 5s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91. -1 unit 68m 6s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 183m 57s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.TestFsck   hadoop.hdfs.TestSafeMode Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781107/HDFS-9574.v3.patch JIRA Issue HDFS-9574 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 09133d95bfa6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 89022f8 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14062/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14062/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14062/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          The failed test cases all pass when run locally.

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.datanode.TestBlockScanner
          Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.947 sec - in org.apache.hadoop.hdfs.server.datanode.TestBlockScanner
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32 sec - in org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.namenode.TestFsck
          Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 135.155 sec - in org.apache.hadoop.hdfs.server.namenode.TestFsck
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.697 sec - in org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
          Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 62.252 sec
           - in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
          Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0
          Running org.apache.hadoop.hdfs.TestSafeMode
          Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.149 sec - in org.apache.hadoop.hdfs.TestSafeMode
          
          Results :
          
          Tests run: 59, Failures: 0, Errors: 0, Skipped: 0
          

          The whitespace warning is from the context, not my change. Nothing to be done for the checkstyle warnings.

          Show
          kihwal Kihwal Lee added a comment - The failed test cases all pass when run locally. ------------------------------------------------------- T E S T S ------------------------------------------------------- Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.datanode.TestBlockScanner Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.947 sec - in org.apache.hadoop.hdfs.server.datanode.TestBlockScanner Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32 sec - in org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.namenode.TestFsck Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 135.155 sec - in org.apache.hadoop.hdfs.server.namenode.TestFsck Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.697 sec - in org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 62.252 sec - in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; support was removed in 8.0 Running org.apache.hadoop.hdfs.TestSafeMode Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.149 sec - in org.apache.hadoop.hdfs.TestSafeMode Results : Tests run: 59, Failures: 0, Errors: 0, Skipped: 0 The whitespace warning is from the context, not my change. Nothing to be done for the checkstyle warnings.
          Hide
          daryn Daryn Sharp added a comment -

          +1 Looks good, few suggestions if you think they would add value, up to you.

          DFSInputStream: Instead of tracking the retryList separately, would it be easier to just add it back to the nodeList and set the isRetry boolean?

          DataXceiver: Might consider changing checkAccess to not require the stream and just have it call getBufferedOutputStream.

          Show
          daryn Daryn Sharp added a comment - +1 Looks good, few suggestions if you think they would add value, up to you. DFSInputStream : Instead of tracking the retryList separately, would it be easier to just add it back to the nodeList and set the isRetry boolean? DataXceiver : Might consider changing checkAccess to not require the stream and just have it call getBufferedOutputStream .
          Hide
          kihwal Kihwal Lee added a comment - - edited

          DFSInputStream: Instead of tracking the retryList separately, would it be easier to just add it back to the nodeList and set the isRetry boolean?

          I thought about doing that. But with one list, it is hard to tell whether it has any good candidate left to try or it has only retriable nodes. The code ended up being more complicated than necessary so I settled with two separate lists.

          DataXceiver: Might consider changing checkAccess to not require the stream and just have it call getBufferedOutputStream.

          It looks like that might be okay for the current usages. I didn't try to improve all existing inconsistencies in there. It might be better to be done in a separate clean-up jira.

          Show
          kihwal Kihwal Lee added a comment - - edited DFSInputStream: Instead of tracking the retryList separately, would it be easier to just add it back to the nodeList and set the isRetry boolean? I thought about doing that. But with one list, it is hard to tell whether it has any good candidate left to try or it has only retriable nodes. The code ended up being more complicated than necessary so I settled with two separate lists. DataXceiver: Might consider changing checkAccess to not require the stream and just have it call getBufferedOutputStream. It looks like that might be okay for the current usages. I didn't try to improve all existing inconsistencies in there. It might be better to be done in a separate clean-up jira.
          Hide
          kihwal Kihwal Lee added a comment -

          Committed to trunk, branch-2 and branch-2.8. branch-2.7 and 2.6 need a separate patch since files have been moved and the hdfs client has been separated out.

          Show
          kihwal Kihwal Lee added a comment - Committed to trunk, branch-2 and branch-2.8. branch-2.7 and 2.6 need a separate patch since files have been moved and the hdfs client has been separated out.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9073 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9073/)
          HDFS-9574. Reduce client failures during datanode restart. Contributed (kihwal: rev 38c4c14472996562eb3d610649246770c2888c6b)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataXceiverLazyPersistHint.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9073 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9073/ ) HDFS-9574 . Reduce client failures during datanode restart. Contributed (kihwal: rev 38c4c14472996562eb3d610649246770c2888c6b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataXceiverLazyPersistHint.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          Hide
          kihwal Kihwal Lee added a comment -

          The patch for branch-2.7 is not very different from the trunk/branch-2 one.

          • Simple context differences.
          • Difference due to hdfs client split and file moves.
          • AccessMode moved from BlockTokenSecretManager to BlockTokenIdentifier.
          Show
          kihwal Kihwal Lee added a comment - The patch for branch-2.7 is not very different from the trunk/branch-2 one. Simple context differences. Difference due to hdfs client split and file moves. AccessMode moved from BlockTokenSecretManager to BlockTokenIdentifier .
          Hide
          kihwal Kihwal Lee added a comment -

          The 2.6 patch is identical to the 2.7 patch except, these two minor differences.

          • StopWatch is not available, so Time.monotonicNow() was used instead.
          • DataNodeFaultInjector does not have set() method. instance is directly set in the test case.
          Show
          kihwal Kihwal Lee added a comment - The 2.6 patch is identical to the 2.7 patch except, these two minor differences. StopWatch is not available, so Time.monotonicNow() was used instead. DataNodeFaultInjector does not have set() method. instance is directly set in the test case.
          Hide
          kihwal Kihwal Lee added a comment -

          This is an important improvement for rolling upgrades. Committed to branch-2.7 and branch-2.6.

          Show
          kihwal Kihwal Lee added a comment - This is an important improvement for rolling upgrades. Committed to branch-2.7 and branch-2.6.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          This is an important improvement for rolling upgrades. Committed to branch-2.7 and branch-2.6.

          +1 for this after the fact.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - This is an important improvement for rolling upgrades. Committed to branch-2.7 and branch-2.6. +1 for this after the fact.
          Hide
          kihwal Kihwal Lee added a comment -

          Arpit Agarwal, please elaborate the reason for marking it incompatible. If valid, we need to update the release note as well.

          Show
          kihwal Kihwal Lee added a comment - Arpit Agarwal , please elaborate the reason for marking it incompatible. If valid, we need to update the release note as well.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Kihwal Lee, my apologies. That was a mistake, thanks for catching it.

          Show
          arpitagarwal Arpit Agarwal added a comment - Kihwal Lee , my apologies. That was a mistake, thanks for catching it.

            People

            • Assignee:
              kihwal Kihwal Lee
              Reporter:
              kihwal Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development