Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7284

Add more debug info to BlockInfoUnderConstruction#setGenerationStampAndVerifyReplicas

    Details

      Description

      When I was looking at some replica loss issue, I got the following info from log

      2014-10-13 01:54:53,104 INFO BlockStateChange: BLOCK* Removing stale replica from location x.x.x.x
      

      I could just know that a replica is removed, but I don't know which block and its timestamp. I need to know the id and timestamp of the block from the log file.
      So it's better to add more info including block id and timestamp to the code snippet

          for (ReplicaUnderConstruction r : replicas) {
            if (genStamp != r.getGenerationStamp()) {
              r.getExpectedLocation().removeBlock(this);
              NameNode.blockStateChangeLog.info("BLOCK* Removing stale replica "
                  + "from location: " + r.getExpectedLocation());
            }
          }
      
      1. HDFS-7284.001.patch
        0.9 kB
        Wei-Chiu Chuang
      2. HDFS-7284.002.patch
        1 kB
        Wei-Chiu Chuang
      3. HDFS-7284.003.patch
        3 kB
        Wei-Chiu Chuang
      4. HDFS-7284.004.patch
        2 kB
        Wei-Chiu Chuang
      5. HDFS-7284.005.patch
        2 kB
        Wei-Chiu Chuang

        Activity

        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Hi Hu,
        Thanks for reporting the issue. I'm interested in working on it and am assigning it to myself.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Hi Hu, Thanks for reporting the issue. I'm interested in working on it and am assigning it to myself.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        A small patch to improve diagnostic message per the request.
        I am not adding new test cases, but I can confirm that existing test cases (specifically TestBlockTokenWithDFS.testWrite) shows the new messages when log4j's log level is set to DEBUG)

        Show
        jojochuang Wei-Chiu Chuang added a comment - A small patch to improve diagnostic message per the request. I am not adding new test cases, but I can confirm that existing test cases (specifically TestBlockTokenWithDFS.testWrite) shows the new messages when log4j's log level is set to DEBUG)
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 15m 47s Findbugs (version ) appears to be broken on trunk.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 45s There were no new javac warning messages.
        +1 javadoc 10m 1s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 36s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 7s Pre-build of native portion
        -1 hdfs tests 137m 12s Tests failed in hadoop-hdfs.
            179m 34s  



        Reason Tests
        Failed unit tests hadoop.hdfs.web.TestWebHDFSOAuth2
        Timed out tests org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12762053/HDFS-7284.001.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / e52bc69
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12659/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12659/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12659/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 15m 47s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 45s There were no new javac warning messages. +1 javadoc 10m 1s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 36s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 7s Pre-build of native portion -1 hdfs tests 137m 12s Tests failed in hadoop-hdfs.     179m 34s   Reason Tests Failed unit tests hadoop.hdfs.web.TestWebHDFSOAuth2 Timed out tests org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12762053/HDFS-7284.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e52bc69 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12659/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12659/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12659/console This message was automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Hu Liu, for reporting the issue and Wei-Chiu Chuang for the patch. I think this is a good supportability jira.

        Hi Wei-Chiu Chuang,

        I think we can change the message to:

            NameNode.blockStateChangeLog.debug("BLOCK* Removing stale replica "
                  + r + " of block " + ((Block)r));
        

        which utilize the toString methods of ReplicaUnderConstruction and Block classes to provide needed info (block id, timestamp), plus replica state which would help in cases.

        The only thing is, if people used to search for "from location: " string for this message, it won't be there.
        But I think this is ok, since this is a debug message and the keyword "Remove stale replica" is not changed.

        Wonder what other people would think. Hi Andrew Wang, mind comment on my comment?

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Hu Liu, for reporting the issue and Wei-Chiu Chuang for the patch. I think this is a good supportability jira. Hi Wei-Chiu Chuang , I think we can change the message to: NameNode.blockStateChangeLog.debug( "BLOCK* Removing stale replica " + r + " of block " + ((Block)r)); which utilize the toString methods of ReplicaUnderConstruction and Block classes to provide needed info (block id, timestamp), plus replica state which would help in cases. The only thing is, if people used to search for "from location: " string for this message, it won't be there. But I think this is ok, since this is a debug message and the keyword "Remove stale replica" is not changed. Wonder what other people would think. Hi Andrew Wang , mind comment on my comment? Thanks.
        Hide
        andrew.wang Andrew Wang added a comment -

        Sure, using the built-in toStrings is a good idea. I will say though that the block state change log is super busy as it is, so making the logs concise is good.

        Debug logging can also still have a performance impact. I think this logger was switched over to SLF4J, so let's make sure to use the template-style logging to avoid unnecessary string construction costs.

        Show
        andrew.wang Andrew Wang added a comment - Sure, using the built-in toStrings is a good idea. I will say though that the block state change log is super busy as it is, so making the logs concise is good. Debug logging can also still have a performance impact. I think this logger was switched over to SLF4J, so let's make sure to use the template-style logging to avoid unnecessary string construction costs.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Cool! Thanks Andrew Wang. Learned a new lesson.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Cool! Thanks Andrew Wang . Learned a new lesson.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Andrew Wang, nice comments.

        I realize I made a mistake, there is no easy way to call Block#toString() from object r of ReplicaUnderConstruction class, doing a cast like I suggested earlier wouldn't work.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Andrew Wang , nice comments. I realize I made a mistake, there is no easy way to call Block#toString() from object r of ReplicaUnderConstruction class, doing a cast like I suggested earlier wouldn't work.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Rev2: Use parameterized log

        Show
        jojochuang Wei-Chiu Chuang added a comment - Rev2: Use parameterized log
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Wei-Chiu Chuang,

        It's important to have consistent block name appear in the log, so people can analyze the actions happened to a given block across the board by searching for the "blk_id_timestamp" or "blk_id".

        I'd suggest adding the following code to Block.java:

          /**
           */
          public static String toString(final Block b) {
            return b.getBlockName() + "_" + b.getGenerationStamp();
          }
          
          /**
           */
          @Override
          public String toString() {
            return toString(this);
          }
        

        and change the message you are working on to

             NameNode.blockStateChangeLog.debug("BLOCK* Removing stale replica {}"
        	          + " of {}", r, Block.toString(r));
        

        Hi Andrew Wang, does this sound good to you? I think the replica state that comes with ReplicaUnderConstruction#toString would help debugging.

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Wei-Chiu Chuang , It's important to have consistent block name appear in the log, so people can analyze the actions happened to a given block across the board by searching for the "blk_id_timestamp" or "blk_id". I'd suggest adding the following code to Block.java: /** */ public static String toString( final Block b) { return b.getBlockName() + "_" + b.getGenerationStamp(); } /** */ @Override public String toString() { return toString( this ); } and change the message you are working on to NameNode.blockStateChangeLog.debug( "BLOCK* Removing stale replica {}" + " of {}" , r, Block.toString(r)); Hi Andrew Wang , does this sound good to you? I think the replica state that comes with ReplicaUnderConstruction#toString would help debugging. Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 18m 25s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 59s There were no new javac warning messages.
        +1 javadoc 10m 34s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 25s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 11s Pre-build of native portion
        -1 hdfs tests 50m 12s Tests failed in hadoop-hdfs.
            96m 53s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
          hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
          hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
          hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
          hadoop.hdfs.server.namenode.snapshot.TestSnapshot
          hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport
        Timed out tests org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          org.apache.hadoop.hdfs.server.namenode.TestFileTruncate



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768308/HDFS-7284.002.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 35a303d
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13153/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13153/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13153/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13153/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 25s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 59s There were no new javac warning messages. +1 javadoc 10m 34s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 25s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 11s Pre-build of native portion -1 hdfs tests 50m 12s Tests failed in hadoop-hdfs.     96m 53s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap   hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.namenode.snapshot.TestSnapshot   hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport Timed out tests org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   org.apache.hadoop.hdfs.server.namenode.TestFileTruncate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768308/HDFS-7284.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 35a303d Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13153/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13153/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13153/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13153/console This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Yongjun Zhang Good idea!
        The output of your suggested change is:
        2015-10-23 10:21:18,647 [IPC Server handler 7 on 51002] DEBUG BlockStateChange (BlockInfo.java:setGenerationStampAndVerifyReplicas(396)) - BLOCK* Removing stale replica ReplicaUC[[DISK]DS-b87b985d-6dc7-448e-9d45-dcd6c2c8ec37:NORMAL:127.0.0.1:51003|RBW] of blk_1073741826_1002

        Attaching rev3 based on Yongjun's suggestion.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Yongjun Zhang Good idea! The output of your suggested change is: 2015-10-23 10:21:18,647 [IPC Server handler 7 on 51002] DEBUG BlockStateChange (BlockInfo.java:setGenerationStampAndVerifyReplicas(396)) - BLOCK* Removing stale replica ReplicaUC[ [DISK] DS-b87b985d-6dc7-448e-9d45-dcd6c2c8ec37:NORMAL:127.0.0.1:51003|RBW] of blk_1073741826_1002 Attaching rev3 based on Yongjun's suggestion.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Wei-Chiu Chuang,

        Thanks for the new rev. I noticed that you changed the default logger setting from info to debug, definitely we need to change it back:

        log4j.rootLogger=debug,stdout

        +1 after that pending jenkins test.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Wei-Chiu Chuang , Thanks for the new rev. I noticed that you changed the default logger setting from info to debug, definitely we need to change it back: log4j.rootLogger=debug,stdout +1 after that pending jenkins test.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks Yongjun Zhang for the code review. I am attaching a new version with no log4 change.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks Yongjun Zhang for the code review. I am attaching a new version with no log4 change.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Sorry one more thing
        Suggest to change:

        * A helper method to output the string representation of a derived class,
        

        to

        * A helper method to output the string representation of the Block portion of
        * a derived class' instance.
        
        Show
        yzhangal Yongjun Zhang added a comment - Sorry one more thing Suggest to change: * A helper method to output the string representation of a derived class, to * A helper method to output the string representation of the Block portion of * a derived class' instance.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks for catching my bad English :/

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks for catching my bad English :/
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 25m 7s Findbugs (version 3.0.0) appears to be broken on trunk.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 12m 57s There were no new javac warning messages.
        +1 javadoc 14m 22s There were no new javadoc warning messages.
        +1 release audit 0m 38s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 3m 4s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 2m 37s mvn install still works.
        +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse.
        -1 findbugs 6m 32s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
        +1 native 4m 16s Pre-build of native portion
        -1 hdfs tests 56m 48s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 33s Tests passed in hadoop-hdfs-client.
            127m 44s  



        Reason Tests
        FindBugs module:hadoop-hdfs
        Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
          hadoop.hdfs.server.namenode.ha.TestEditLogTailer
          hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
        Timed out tests org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager
          org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
          org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
          org.apache.hadoop.hdfs.server.mover.TestStorageMover



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768378/HDFS-7284.005.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 600ad7b
        Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13162/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13162/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13162/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13162/testReport/
        Java 1.7.0_55
        uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13162/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 25m 7s Findbugs (version 3.0.0) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 12m 57s There were no new javac warning messages. +1 javadoc 14m 22s There were no new javadoc warning messages. +1 release audit 0m 38s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 3m 4s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 2m 37s mvn install still works. +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse. -1 findbugs 6m 32s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 4m 16s Pre-build of native portion -1 hdfs tests 56m 48s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 33s Tests passed in hadoop-hdfs-client.     127m 44s   Reason Tests FindBugs module:hadoop-hdfs Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes Timed out tests org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean   org.apache.hadoop.hdfs.server.mover.TestStorageMover Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768378/HDFS-7284.005.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 600ad7b Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13162/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13162/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13162/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13162/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13162/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 18m 23s Findbugs (version 3.0.0) appears to be broken on trunk.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 3s There were no new javac warning messages.
        +1 javadoc 10m 46s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 2m 9s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 40s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        -1 findbugs 4m 38s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
        +1 native 3m 12s Pre-build of native portion
        -1 hdfs tests 0m 24s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 29s Tests passed in hadoop-hdfs-client.
            50m 46s  



        Reason Tests
        FindBugs module:hadoop-hdfs
        Failed build hadoop-hdfs



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768378/HDFS-7284.005.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 15eb84b
        Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13165/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13165/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13165/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13165/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13165/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 23s Findbugs (version 3.0.0) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 3s There were no new javac warning messages. +1 javadoc 10m 46s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 9s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 40s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. -1 findbugs 4m 38s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 12s Pre-build of native portion -1 hdfs tests 0m 24s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 29s Tests passed in hadoop-hdfs-client.     50m 46s   Reason Tests FindBugs module:hadoop-hdfs Failed build hadoop-hdfs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768378/HDFS-7284.005.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 15eb84b Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13165/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13165/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13165/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13165/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13165/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 20m 17s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 17s There were no new javac warning messages.
        +1 javadoc 10m 25s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 2m 35s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 40s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 4m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 13s Pre-build of native portion
        -1 hdfs tests 51m 17s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 36s Tests passed in hadoop-hdfs-client.
            103m 55s  



        Reason Tests
        Failed unit tests hadoop.hdfs.util.TestByteArrayManager



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768378/HDFS-7284.005.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 7781fe1
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13178/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13178/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13178/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13178/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13178/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 20m 17s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 17s There were no new javac warning messages. +1 javadoc 10m 25s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 35s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 40s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 13s Pre-build of native portion -1 hdfs tests 51m 17s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 36s Tests passed in hadoop-hdfs-client.     103m 55s   Reason Tests Failed unit tests hadoop.hdfs.util.TestByteArrayManager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768378/HDFS-7284.005.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 7781fe1 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13178/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13178/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/13178/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13178/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13178/console This message was automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        +1 on 005, I will commit momentarily.

        Show
        yzhangal Yongjun Zhang added a comment - +1 on 005, I will commit momentarily.
        Hide
        yzhangal Yongjun Zhang added a comment -

        I committed to trunk and branch-2. Thanks Hu Liu, for reporting the issue, and Wei-Chiu for the contribution!

        Show
        yzhangal Yongjun Zhang added a comment - I committed to trunk and branch-2. Thanks Hu Liu, for reporting the issue, and Wei-Chiu for the contribution!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8710 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8710/)
        HDFS-7284. Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8710 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8710/ ) HDFS-7284 . Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1323 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1323/)
        HDFS-7284. Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1323 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1323/ ) HDFS-7284 . Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #599 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/599/)
        HDFS-7284. Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #599 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/599/ ) HDFS-7284 . Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2477/)
        HDFS-7284. Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2477/ ) HDFS-7284 . Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2530 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2530/)
        HDFS-7284. Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2530 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2530/ ) HDFS-7284 . Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #540 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/540/)
        HDFS-7284. Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #540 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/540/ ) HDFS-7284 . Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #588 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/588/)
        HDFS-7284. Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #588 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/588/ ) HDFS-7284 . Add more debug info to (yzhang: rev 5e718de522328d1112ad38063596c204aa43f539) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java

          People

          • Assignee:
            jojochuang Wei-Chiu Chuang
            Reporter:
            huLiu Hu Liu,
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development