Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9706

Log more details in debug logs in BlockReceiver's constructor

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Currently BlockReceiver's constructor has some debug logs to help identifying problems. During my triage of HDFS-9701, I needed to add the isTransfer into the logs to see which block the code goes later.

      I propose to add more details in the debug logs, to save future effort. Will also see whether more details need to be logged.

      1. HDFS-9706.01.patch
        0.9 kB
        Xiao Chen
      2. HDFS-9706.02.patch
        2 kB
        Xiao Chen
      3. HDFS-9706.03.patch
        2 kB
        Xiao Chen

        Activity

        Hide
        xiaochen Xiao Chen added a comment -

        Patch 1 adds isTransfer, and some other infos I think might be useful.

        Show
        xiaochen Xiao Chen added a comment - Patch 1 adds isTransfer, and some other infos I think might be useful.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 9m 47s trunk passed
        +1 compile 1m 11s trunk passed with JDK v1.8.0_66
        +1 compile 0m 57s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 1m 9s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 2m 21s trunk passed
        +1 javadoc 1m 35s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 22s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 1s the patch passed
        +1 compile 1m 5s the patch passed with JDK v1.8.0_66
        +1 javac 1m 5s the patch passed
        +1 compile 0m 52s the patch passed with JDK v1.7.0_91
        +1 javac 0m 52s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 1m 8s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 39s the patch passed
        +1 javadoc 1m 34s the patch passed with JDK v1.8.0_66
        +1 javadoc 2m 17s the patch passed with JDK v1.7.0_91
        -1 unit 124m 58s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 148m 22s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 1m 22s Patch does not generate ASF License warnings.
        309m 39s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
          hadoop.hdfs.server.namenode.ha.TestEditLogTailer
          hadoop.hdfs.security.TestDelegationTokenForProxyUser
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
          hadoop.hdfs.TestDatanodeRegistration
          hadoop.hdfs.server.datanode.TestBlockReplacement
          hadoop.hdfs.server.namenode.ha.TestHAAppend
          hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.server.datanode.TestDirectoryScanner
        JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage
          hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          hadoop.hdfs.server.namenode.ha.TestEditLogTailer
          hadoop.hdfs.TestPersistBlocks
          hadoop.hdfs.security.TestDelegationTokenForProxyUser
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.datanode.TestBlockReplacement
          hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
          hadoop.hdfs.qjournal.TestSecureNNWithQJM
          hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.TestEncryptionZones
          hadoop.hdfs.server.namenode.TestDeleteRace
          hadoop.hdfs.server.datanode.TestDirectoryScanner
        JDK v1.7.0_91 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784457/HDFS-9706.01.patch
        JIRA Issue HDFS-9706
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux bf507899add0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d0d7c22
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14248/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 77MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14248/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 47s trunk passed +1 compile 1m 11s trunk passed with JDK v1.8.0_66 +1 compile 0m 57s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 25s trunk passed +1 mvnsite 1m 9s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 2m 21s trunk passed +1 javadoc 1m 35s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 22s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 1s the patch passed +1 compile 1m 5s the patch passed with JDK v1.8.0_66 +1 javac 1m 5s the patch passed +1 compile 0m 52s the patch passed with JDK v1.7.0_91 +1 javac 0m 52s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 1m 8s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 39s the patch passed +1 javadoc 1m 34s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 17s the patch passed with JDK v1.7.0_91 -1 unit 124m 58s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 148m 22s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 1m 22s Patch does not generate ASF License warnings. 309m 39s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_91 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784457/HDFS-9706.01.patch JIRA Issue HDFS-9706 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bf507899add0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d0d7c22 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14248/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14248/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14248/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        This is a logging change, so no tests added. The jenkins failures seems unrelated.

        Show
        xiaochen Xiao Chen added a comment - This is a logging change, so no tests added. The jenkins failures seems unrelated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Xiao Chen,

        Thanks for working on this. Couple of suggestions:

        • Instead of printing isTransfer, it may be better just to print stage's value.
        • We are printing almost all parameters except 3 or 4 others. I'd suggest that we take this opportunity to make a complete printing, and order it as how the parameters are passed to the method. (that is, one day we may need to know the value of the other parameters when investigating a new issue),

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Xiao Chen , Thanks for working on this. Couple of suggestions: Instead of printing isTransfer , it may be better just to print stage 's value. We are printing almost all parameters except 3 or 4 others. I'd suggest that we take this opportunity to make a complete printing, and order it as how the parameters are passed to the method. (that is, one day we may need to know the value of the other parameters when investigating a new issue), Thanks.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Yongjun for the comments.
        I agree with your suggestions. Patch 2 rewrote the debug log, to match the parameters passed in. After that, I additionally printed isClient, isDatanode, and responseInterval. These 3 can be deducted from input or read from the config of the input datanode, but since this is debug log, I think having 1 more line to save some developer time is a good tradeoff.

        Below is a sample output of the log:

        2016-01-27 09:34:12,849 [DataXceiver for client cl at /127.0.0.1:50088 [Receiving block BP-908057294-192.168.1.79-1453916050495:blk_1073741825_1001]] DEBUG datanode.DataNode (BlockReceiver.java:<init>(189)) - BlockReceiver: BP-908057294-192.168.1.79-1453916050495:blk_1073741825_1001
         storageType=DISK, inAddr=/127.0.0.1:50088
         myAddr=/127.0.0.1:50079, stage=DATA_STREAMING, newGs=0
         minBytesRcvd=1
         maxBytesRcvd=1, clientname=cl
         srcDataNode=:0
         datanode=127.0.0.1:50079
         requestedChecksum=DataChecksum(type=CRC32C, chunkSize=512)
         cachingStrategy=CachingStrategy(dropBehind=null, readahead=null)
         allowLazyPersist=false, pinning=false
         isClient=true, isDatanode=false
         responseInterval=30000
        
        Show
        xiaochen Xiao Chen added a comment - Thanks Yongjun for the comments. I agree with your suggestions. Patch 2 rewrote the debug log, to match the parameters passed in. After that, I additionally printed isClient , isDatanode , and responseInterval . These 3 can be deducted from input or read from the config of the input datanode, but since this is debug log, I think having 1 more line to save some developer time is a good tradeoff. Below is a sample output of the log: 2016-01-27 09:34:12,849 [DataXceiver for client cl at /127.0.0.1:50088 [Receiving block BP-908057294-192.168.1.79-1453916050495:blk_1073741825_1001]] DEBUG datanode.DataNode (BlockReceiver.java:<init>(189)) - BlockReceiver: BP-908057294-192.168.1.79-1453916050495:blk_1073741825_1001 storageType=DISK, inAddr=/127.0.0.1:50088 myAddr=/127.0.0.1:50079, stage=DATA_STREAMING, newGs=0 minBytesRcvd=1 maxBytesRcvd=1, clientname=cl srcDataNode=:0 datanode=127.0.0.1:50079 requestedChecksum=DataChecksum(type=CRC32C, chunkSize=512) cachingStrategy=CachingStrategy(dropBehind=null, readahead=null) allowLazyPersist=false, pinning=false isClient=true, isDatanode=false responseInterval=30000
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Xiao for the new rev. One further suggestion is, we can put multiple entries to the same line so we have fewer lines in the log, what about:

        2016-01-27 09:34:12,849 [DataXceiver for client cl at /127.0.0.1:50088 [Receiving block BP-908057294-192.168.1.79-1453916050495:blk_1073741825_1001]] DEBUG datanode.DataNode (BlockReceiver.java:<init>(189)) - BlockReceiver: BP-908057294-192.168.1.79-1453916050495:blk_1073741825_1001
         storageType=DISK, inAddr=/127.0.0.1:50088, myAddr=/127.0.0.1:50079, 
         stage=DATA_STREAMING, newGs=0, minBytesRcvd=1, maxBytesRcvd=1,
         clientname=cl, srcDataNode=:0, datanode=127.0.0.1:50079
         requestedChecksum=DataChecksum(type=CRC32C, chunkSize=512), cachingStrategy=CachingStrategy(dropBehind=null, readahead=null)
         allowLazyPersist=false, pinning=false, isClient=true, isDatanode=false, responseInterval=30000
        

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Xiao for the new rev. One further suggestion is, we can put multiple entries to the same line so we have fewer lines in the log, what about: 2016-01-27 09:34:12,849 [DataXceiver for client cl at /127.0.0.1:50088 [Receiving block BP-908057294-192.168.1.79-1453916050495:blk_1073741825_1001]] DEBUG datanode.DataNode (BlockReceiver.java:<init>(189)) - BlockReceiver: BP-908057294-192.168.1.79-1453916050495:blk_1073741825_1001 storageType=DISK, inAddr=/127.0.0.1:50088, myAddr=/127.0.0.1:50079, stage=DATA_STREAMING, newGs=0, minBytesRcvd=1, maxBytesRcvd=1, clientname=cl, srcDataNode=:0, datanode=127.0.0.1:50079 requestedChecksum=DataChecksum(type=CRC32C, chunkSize=512), cachingStrategy=CachingStrategy(dropBehind= null , readahead= null ) allowLazyPersist= false , pinning= false , isClient= true , isDatanode= false , responseInterval=30000 Thanks.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Yongjun for the advice, makes sense to me. Attaching patch 3 per suggested.

        Show
        xiaochen Xiao Chen added a comment - Thanks Yongjun for the advice, makes sense to me. Attaching patch 3 per suggested.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Xiao for the new rev. +1 pending jenkins.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Xiao for the new rev. +1 pending jenkins.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 9m 12s trunk passed
        +1 compile 0m 54s trunk passed with JDK v1.8.0_66
        +1 compile 0m 46s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 1m 2s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 2m 20s trunk passed
        +1 javadoc 1m 30s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 7s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 57s the patch passed
        +1 compile 0m 50s the patch passed with JDK v1.8.0_66
        +1 javac 0m 50s the patch passed
        +1 compile 0m 48s the patch passed with JDK v1.7.0_91
        +1 javac 0m 48s the patch passed
        +1 checkstyle 0m 20s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 57 unchanged - 1 fixed = 57 total (was 58)
        +1 mvnsite 0m 58s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 39s the patch passed
        +1 javadoc 1m 16s the patch passed with JDK v1.8.0_66
        +1 javadoc 2m 6s the patch passed with JDK v1.7.0_91
        -1 unit 67m 6s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 55m 12s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 26s Patch does not generate ASF License warnings.
        154m 15s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
          hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.TestEncryptionZones
          hadoop.hdfs.TestFileAppend
          hadoop.hdfs.shortcircuit.TestShortCircuitCache
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784683/HDFS-9706.02.patch
        JIRA Issue HDFS-9706
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f1f4ea9db466 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 97056c3
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14261/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14261/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14261/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14261/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14261/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 77MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14261/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 12s trunk passed +1 compile 0m 54s trunk passed with JDK v1.8.0_66 +1 compile 0m 46s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 24s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 2m 20s trunk passed +1 javadoc 1m 30s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 7s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 57s the patch passed +1 compile 0m 50s the patch passed with JDK v1.8.0_66 +1 javac 0m 50s the patch passed +1 compile 0m 48s the patch passed with JDK v1.7.0_91 +1 javac 0m 48s the patch passed +1 checkstyle 0m 20s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 57 unchanged - 1 fixed = 57 total (was 58) +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 39s the patch passed +1 javadoc 1m 16s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 6s the patch passed with JDK v1.7.0_91 -1 unit 67m 6s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 55m 12s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 154m 15s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.TestFileAppend   hadoop.hdfs.shortcircuit.TestShortCircuitCache JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784683/HDFS-9706.02.patch JIRA Issue HDFS-9706 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f1f4ea9db466 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 97056c3 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14261/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14261/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14261/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14261/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14261/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14261/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Yongjun!
        Failed tests look unrelated, and as mentioned earlier, 'This is a logging change, so no tests added'.

        Show
        xiaochen Xiao Chen added a comment - Thanks Yongjun! Failed tests look unrelated, and as mentioned earlier, 'This is a logging change, so no tests added'.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 8m 19s trunk passed
        +1 compile 0m 42s trunk passed with JDK v1.8.0_66
        +1 compile 0m 44s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 53s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 58s trunk passed
        +1 javadoc 1m 7s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 52s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 38s the patch passed with JDK v1.8.0_66
        +1 javac 0m 38s the patch passed
        +1 compile 0m 42s the patch passed with JDK v1.7.0_91
        +1 javac 0m 42s the patch passed
        +1 checkstyle 0m 20s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 57 unchanged - 1 fixed = 57 total (was 58)
        +1 mvnsite 0m 52s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 8s the patch passed
        +1 javadoc 1m 5s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 48s the patch passed with JDK v1.7.0_91
        -1 unit 67m 27s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 68m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        163m 50s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
          hadoop.hdfs.server.balancer.TestBalancer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784684/HDFS-9706.02.patch
        JIRA Issue HDFS-9706
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 6e4d7280d196 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 97056c3
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14262/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14262/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14262/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14262/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14262/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14262/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 19s trunk passed +1 compile 0m 42s trunk passed with JDK v1.8.0_66 +1 compile 0m 44s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 58s trunk passed +1 javadoc 1m 7s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 52s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 47s the patch passed +1 compile 0m 38s the patch passed with JDK v1.8.0_66 +1 javac 0m 38s the patch passed +1 compile 0m 42s the patch passed with JDK v1.7.0_91 +1 javac 0m 42s the patch passed +1 checkstyle 0m 20s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 57 unchanged - 1 fixed = 57 total (was 58) +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 8s the patch passed +1 javadoc 1m 5s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 48s the patch passed with JDK v1.7.0_91 -1 unit 67m 27s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 68m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 163m 50s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.server.balancer.TestBalancer Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784684/HDFS-9706.02.patch JIRA Issue HDFS-9706 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6e4d7280d196 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 97056c3 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14262/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14262/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14262/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14262/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14262/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14262/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 8m 27s trunk passed
        +1 compile 0m 47s trunk passed with JDK v1.8.0_66
        +1 compile 0m 49s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 1m 2s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 2m 9s trunk passed
        +1 javadoc 1m 27s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 7s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 50s the patch passed
        +1 compile 0m 50s the patch passed with JDK v1.8.0_66
        +1 javac 0m 50s the patch passed
        +1 compile 0m 48s the patch passed with JDK v1.7.0_91
        +1 javac 0m 48s the patch passed
        +1 checkstyle 0m 21s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 57 unchanged - 1 fixed = 57 total (was 58)
        +1 mvnsite 0m 55s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 18s the patch passed
        +1 javadoc 1m 18s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 55s the patch passed with JDK v1.7.0_91
        -1 unit 59m 33s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        +1 unit 50m 17s hadoop-hdfs in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        139m 38s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestRecoverStripedFile
          hadoop.hdfs.server.datanode.TestBlockScanner



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784688/HDFS-9706.03.patch
        JIRA Issue HDFS-9706
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 3217268e4756 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 97056c3
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14263/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14263/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14263/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14263/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 27s trunk passed +1 compile 0m 47s trunk passed with JDK v1.8.0_66 +1 compile 0m 49s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 23s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 9s trunk passed +1 javadoc 1m 27s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 7s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 50s the patch passed +1 compile 0m 50s the patch passed with JDK v1.8.0_66 +1 javac 0m 50s the patch passed +1 compile 0m 48s the patch passed with JDK v1.7.0_91 +1 javac 0m 48s the patch passed +1 checkstyle 0m 21s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 57 unchanged - 1 fixed = 57 total (was 58) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 18s the patch passed +1 javadoc 1m 18s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 55s the patch passed with JDK v1.7.0_91 -1 unit 59m 33s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 50m 17s hadoop-hdfs in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 139m 38s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestRecoverStripedFile   hadoop.hdfs.server.datanode.TestBlockScanner Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784688/HDFS-9706.03.patch JIRA Issue HDFS-9706 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3217268e4756 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 97056c3 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14263/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14263/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14263/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14263/console This message was automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Committed to trunk, branch-2, branch-2.8.

        Thanks Xiao for the contribution.

        Show
        yzhangal Yongjun Zhang added a comment - Committed to trunk, branch-2, branch-2.8. Thanks Xiao for the contribution.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9206 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9206/)
        HDFS-9706. Log more details in debug logs in BlockReceiver's (yzhang: rev 8ee060311c89b7faa71dd039481a97ba15e2413d)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9206 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9206/ ) HDFS-9706 . Log more details in debug logs in BlockReceiver's (yzhang: rev 8ee060311c89b7faa71dd039481a97ba15e2413d) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        xiaochen Xiao Chen added a comment -

        Thank you Yongjun for the review and commit!

        Show
        xiaochen Xiao Chen added a comment - Thank you Yongjun for the review and commit!

          People

          • Assignee:
            xiaochen Xiao Chen
            Reporter:
            xiaochen Xiao Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development