Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7235

DataNode#transferBlock should report blocks that don't exist using reportBadBlock

    Details

    • Target Version/s:

      Description

      When to decommission a DN, the process hangs.

      What happens is, when NN chooses a replica as a source to replicate data on the to-be-decommissioned DN to other DNs, it favors choosing this DN to-be-decommissioned as the source of transfer (see BlockManager.java). However, because of the bad disk, the DN would detect the source block to be transfered as invalidBlock with the following logic in FsDatasetImpl.java:

      /** Does the block exist and have the given state? */
        private boolean isValid(final ExtendedBlock b, final ReplicaState state) {
          final ReplicaInfo replicaInfo = volumeMap.get(b.getBlockPoolId(), 
              b.getLocalBlock());
          return replicaInfo != null
              && replicaInfo.getState() == state
              && replicaInfo.getBlockFile().exists();
        }
      

      The reason that this method returns false (detecting invalid block) is because the block file doesn't exist due to bad disk in this case.

      The key issue we found here is, after DN detects an invalid block for the above reason, it doesn't report the invalid block back to NN, thus NN doesn't know that the block is corrupted, and keeps sending the data transfer request to the same DN to be decommissioned, again and again. This caused an infinite loop, so the decommission process hangs.

      Thanks Harsh J for reporting the issue and initial analysis.

      1. HDFS-7235.001.patch
        11 kB
        Yongjun Zhang
      2. HDFS-7235.002.patch
        8 kB
        Yongjun Zhang
      3. HDFS-7235.003.patch
        8 kB
        Yongjun Zhang
      4. HDFS-7235.004.patch
        21 kB
        Yongjun Zhang
      5. HDFS-7235.005.patch
        20 kB
        Yongjun Zhang
      6. HDFS-7235.006.patch
        20 kB
        Yongjun Zhang
      7. HDFS-7235.007.patch
        20 kB
        Yongjun Zhang
      8. HDFS-7235.007.patch
        20 kB
        Yongjun Zhang

        Activity

        Hide
        yzhangal Yongjun Zhang added a comment -

        Submit patch rev 001. The solution is to make DN to reports back to NN the bad block caused by missing block file, thus breaking the reported infinite loop.

        Traced the new test in eclipse, and see the DN does report back to NN the bad block caused by missing block file back.

        Thanks for reviewing.

        Show
        yzhangal Yongjun Zhang added a comment - Submit patch rev 001. The solution is to make DN to reports back to NN the bad block caused by missing block file, thus breaking the reported infinite loop. Traced the new test in eclipse, and see the DN does report back to NN the bad block caused by missing block file back. Thanks for reviewing.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12674341/HDFS-7235.001.patch
        against trunk revision a5ec3d0.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        -1 release audit. The applied patch generated 1 release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestLeaseRecovery2
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8400//testReport/
        Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8400//artifact/patchprocess/patchReleaseAuditProblems.txt
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8400//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674341/HDFS-7235.001.patch against trunk revision a5ec3d0. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 release audit . The applied patch generated 1 release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestLeaseRecovery2 org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8400//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8400//artifact/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8400//console This message is automatically generated.
        Hide
        cmccabe Colin P. McCabe added a comment -

        Thanks for looking at this, Yongjun.

        I don't understand why we need a new function named FsDatasetSpi#isInvalidBlockDueToNonexistentBlockFile. The JavaDoc for FsDatasetSpi#isValid says that it checks if the block "exist[s] and has the given state" and it's clear from the code that this is what it actually implements.

        We start by calling isValid...

          private void transferBlock(ExtendedBlock block, DatanodeInfo[] xferTargets,
              StorageType[] xferTargetStorageTypes) throws IOException {
            BPOfferService bpos = getBPOSForBlock(block);
            DatanodeRegistration bpReg = getDNRegistrationForBP(block.getBlockPoolId());
        
            if (!data.isValidBlock(block)) {
              // block does not exist or is under-construction
              String errStr = "Can't send invalid block " + block;
              LOG.info(errStr);
        
              bpos.trySendErrorReport(DatanodeProtocol.INVALID_BLOCK, errStr);
              return;
            }
            ...
        

        isValid checks whether the block file exists...

        /** Does the block exist and have the given state? */                                                                
          private boolean isValid(final ExtendedBlock b, final ReplicaState state) {                                           
            final ReplicaInfo replicaInfo = volumeMap.get(b.getBlockPoolId(),                                                  
                b.getLocalBlock());
            return replicaInfo != null                                                                                         
                && replicaInfo.getState() == state                                                                             
                && replicaInfo.getBlockFile().exists();
          }   
        

        So there's no need for a new function. isValid already does what you want.

        The key issue we found here is, after DN detects an invalid block for the above reason, it doesn't report the invalid block back to NN, thus NN doesn't know that the block is corrupted, and keeps sending the data transfer request to the same DN to be decommissioned, again and again. This caused an infinite loop, so the decommission process hangs.

        Is this a problem with BPOfferService#trySendErrorReport? If so, it seems like we should fix it there.

        I can see that BPServiceActor#trySendErrorReport calls NameNodeRpc#errorReport, whereas your patch calls NameNodeRpc#reportBadBlocks. What's the reason for this change, and does it fix the bug described above?

        Show
        cmccabe Colin P. McCabe added a comment - Thanks for looking at this, Yongjun. I don't understand why we need a new function named FsDatasetSpi#isInvalidBlockDueToNonexistentBlockFile . The JavaDoc for FsDatasetSpi#isValid says that it checks if the block "exist[s] and has the given state" and it's clear from the code that this is what it actually implements. We start by calling isValid... private void transferBlock(ExtendedBlock block, DatanodeInfo[] xferTargets, StorageType[] xferTargetStorageTypes) throws IOException { BPOfferService bpos = getBPOSForBlock(block); DatanodeRegistration bpReg = getDNRegistrationForBP(block.getBlockPoolId()); if (!data.isValidBlock(block)) { // block does not exist or is under-construction String errStr = "Can't send invalid block " + block; LOG.info(errStr); bpos.trySendErrorReport(DatanodeProtocol.INVALID_BLOCK, errStr); return ; } ... isValid checks whether the block file exists... /** Does the block exist and have the given state? */ private boolean isValid( final ExtendedBlock b, final ReplicaState state) { final ReplicaInfo replicaInfo = volumeMap.get(b.getBlockPoolId(), b.getLocalBlock()); return replicaInfo != null && replicaInfo.getState() == state && replicaInfo.getBlockFile().exists(); } So there's no need for a new function. isValid already does what you want. The key issue we found here is, after DN detects an invalid block for the above reason, it doesn't report the invalid block back to NN, thus NN doesn't know that the block is corrupted, and keeps sending the data transfer request to the same DN to be decommissioned, again and again. This caused an infinite loop, so the decommission process hangs. Is this a problem with BPOfferService#trySendErrorReport ? If so, it seems like we should fix it there. I can see that BPServiceActor#trySendErrorReport calls NameNodeRpc#errorReport , whereas your patch calls NameNodeRpc#reportBadBlocks . What's the reason for this change, and does it fix the bug described above?
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Colin,

        Thanks a lot for the review.

        The key issue identified for the original symptom was, when a block is detected as invalid by the existing isValid() method, we call SendErrorReport() which just log a message there, and Namenode doesn't do more than logging the message for this call, so NameNode doesn't know the block is bad.

        What I did was, I separate the reasons for isValid to be false to two parts,

        • if it's false because getBlockFile().exists() , call reportBadBlocks, so NameNode will record the bad block for future reference.
        • if it's false because either replicaInfo == null OR replicaInfo.getState() != state, it still calls SendErrorReport() like before. Actually for this case, the state has to be FINALIZED. We don't want to report badBlock for state that's RBW for example.

        If we make the change in SendErrorReport, that means we need to change the behavior of this method, to also call reportBadBlocks from there conditionally, which is not clean to me, because SendErrorReport is supposed to just send error report.

        Wonder if this explanation makes sense to you?

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Colin, Thanks a lot for the review. The key issue identified for the original symptom was, when a block is detected as invalid by the existing isValid() method, we call SendErrorReport() which just log a message there, and Namenode doesn't do more than logging the message for this call, so NameNode doesn't know the block is bad. What I did was, I separate the reasons for isValid to be false to two parts, if it's false because getBlockFile().exists() , call reportBadBlocks, so NameNode will record the bad block for future reference. if it's false because either replicaInfo == null OR replicaInfo.getState() != state, it still calls SendErrorReport() like before. Actually for this case, the state has to be FINALIZED. We don't want to report badBlock for state that's RBW for example. If we make the change in SendErrorReport, that means we need to change the behavior of this method, to also call reportBadBlocks from there conditionally, which is not clean to me, because SendErrorReport is supposed to just send error report. Wonder if this explanation makes sense to you? Thanks.
        Hide
        cmccabe Colin P. McCabe added a comment -

        Thanks for explaining this. If I understand correctly, you want blocks that are not in finalized state to cause trySendErrorReport, but blocks that don't exist or have the wrong length to cause reportBadBlocks. That seems reasonable. One improvement that I would suggest is that you don't need to add a new method to FsDatasetSpi to do that. Just call FsDatasetSpi#getLength. If the block doesn't exist, it will throw an IOException which you can catch. Patch looks good aside from that.

        Show
        cmccabe Colin P. McCabe added a comment - Thanks for explaining this. If I understand correctly, you want blocks that are not in finalized state to cause trySendErrorReport , but blocks that don't exist or have the wrong length to cause reportBadBlocks . That seems reasonable. One improvement that I would suggest is that you don't need to add a new method to FsDatasetSpi to do that. Just call FsDatasetSpi#getLength . If the block doesn't exist, it will throw an IOException which you can catch. Patch looks good aside from that.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Colin P. McCabe,

        Thanks a lot for the input. Yes, I expect trySendErrorReport to be called when the blocks are not in finalized state, and reportBadBlocks to be called when the block file doesn't exist.

        To try what you suggested, when isValidBlock returns false, I still need to check the other conditions are true:

              replicaInfo != null                                                                                         
        && replicaInfo.getState() == FINALIZED   
        

        Right now there is no method to get replicaInfo from the DataNode.java side, except a deprecated method

        @Deprecated
          public Replica getReplica(String bpid, long blockId);
        

        I will just call this method if it's ok to use.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Colin P. McCabe , Thanks a lot for the input. Yes, I expect trySendErrorReport to be called when the blocks are not in finalized state, and reportBadBlocks to be called when the block file doesn't exist. To try what you suggested, when isValidBlock returns false, I still need to check the other conditions are true: replicaInfo != null && replicaInfo.getState() == FINALIZED Right now there is no method to get replicaInfo from the DataNode.java side, except a deprecated method @Deprecated public Replica getReplica( String bpid, long blockId); I will just call this method if it's ok to use.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Colin P. McCabe,

        Thanks for your earlier review. I just uploaded a new rev per what you suggested. (002)

        There is one issue with this approach, the changed code in DataNode now kind of "sees" the FSdatasetImpl implemetation. But maybe it's fine.

        BTW, since I need to get replicaInfo in DataNode, and I need to make sure the replica state is FINALIZED, I simply called the exists() method to check block file existence.

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Colin P. McCabe , Thanks for your earlier review. I just uploaded a new rev per what you suggested. (002) There is one issue with this approach, the changed code in DataNode now kind of "sees" the FSdatasetImpl implemetation. But maybe it's fine. BTW, since I need to get replicaInfo in DataNode, and I need to make sure the replica state is FINALIZED, I simply called the exists() method to check block file existence. Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12674641/HDFS-7235.002.patch
        against trunk revision cc93e7e.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        -1 javac. The applied patch generated 1265 javac compiler warnings (more than the trunk's current 1264 warnings).

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8414//testReport/
        Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8414//artifact/patchprocess/diffJavacWarnings.txt
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8414//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674641/HDFS-7235.002.patch against trunk revision cc93e7e. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. -1 javac . The applied patch generated 1265 javac compiler warnings (more than the trunk's current 1264 warnings). +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8414//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8414//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8414//console This message is automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        HI Colin P. McCabe,

        Thanks for the review and discussion yesterday. I was in a rush to leave when I put the previous comment with patch yesterday. Here is some more info to share:

        • You said that external user might be deriving FsDatasetSpi interface, any change to this interface might imply compatibility. This is a very good point. So indeed it'd be nice if we can avoid changing FsDatasetSpi.
        • If we use FsDatasetSpi#getLength method to check file existence, it's not guaranteed that the replica state is FINALIZED. So it's not sufficient for the fix here.
        • Without changing FsDatasetSpi, we need to add similar logic as I did in rev 001 to DataNode.java. To check replica state in DataNode.java, I had to use the deprecated method getReplica().
        • Having this logic in DataNode.java is a bit concern to me, DataNode is supposed to use FsDatasetSpi interface only, now we incorporate logic specific to FsDatasetImpl in DataNode.java. If user derives FsDatasetSpi and write their own version, the logic may not be the same as FsDatasetImpl. This might cause potential problem. This is the point I was trying to make in last comment.

        Would you please comment again?

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - HI Colin P. McCabe , Thanks for the review and discussion yesterday. I was in a rush to leave when I put the previous comment with patch yesterday. Here is some more info to share: You said that external user might be deriving FsDatasetSpi interface, any change to this interface might imply compatibility. This is a very good point. So indeed it'd be nice if we can avoid changing FsDatasetSpi. If we use FsDatasetSpi#getLength method to check file existence, it's not guaranteed that the replica state is FINALIZED. So it's not sufficient for the fix here. Without changing FsDatasetSpi, we need to add similar logic as I did in rev 001 to DataNode.java. To check replica state in DataNode.java, I had to use the deprecated method getReplica(). Having this logic in DataNode.java is a bit concern to me, DataNode is supposed to use FsDatasetSpi interface only, now we incorporate logic specific to FsDatasetImpl in DataNode.java. If user derives FsDatasetSpi and write their own version, the logic may not be the same as FsDatasetImpl. This might cause potential problem. This is the point I was trying to make in last comment. Would you please comment again? Thanks.
        Hide
        yzhangal Yongjun Zhang added a comment -

        FYI Colin P. McCabe, I found a related jira HDFS-5194 and posted a comment there.

        Show
        yzhangal Yongjun Zhang added a comment - FYI Colin P. McCabe , I found a related jira HDFS-5194 and posted a comment there.
        Hide
        cmccabe Colin P. McCabe added a comment - - edited
        1787      ReplicaInfo replicaInfo = null;
        1788	      synchronized(data) {
        1789	        replicaInfo = (ReplicaInfo) data.getReplica( block.getBlockPoolId(),
        1790	            block.getBlockId());
        1791	      }
        1792	      if (replicaInfo != null 
        1793	          && replicaInfo.getState() == ReplicaState.FINALIZED 
        1794	          && !replicaInfo.getBlockFile().exists()) {
        

        You can't release the lock this way. Once you release the lock, replicaInfo could be mutated at any time. So you need to do all the check under the lock.

        1795                //
        1796	        // Report back to NN bad block caused by non-existent block file.
        1797	        // WATCH-OUT: be sure the conditions checked above matches the following
        1798	        // method in FsDatasetImpl.java:
        1799	        //   boolean isValidBlock(ExtendedBlock b)
        1800	        // all other conditions need to be true except that 
        1801	        // replicaInfo.getBlockFile().exists() returns false.
        1802	        //
        

        I don't think we need the "WATCH-OUT" part. We shouldn't be calling isValidBlock, so why do we care if the check is the same as that check?

        I generally agree with this approach and I think we can get this in if that's fixed.

        Show
        cmccabe Colin P. McCabe added a comment - - edited 1787 ReplicaInfo replicaInfo = null ; 1788 synchronized (data) { 1789 replicaInfo = (ReplicaInfo) data.getReplica( block.getBlockPoolId(), 1790 block.getBlockId()); 1791 } 1792 if (replicaInfo != null 1793 && replicaInfo.getState() == ReplicaState.FINALIZED 1794 && !replicaInfo.getBlockFile().exists()) { You can't release the lock this way. Once you release the lock, replicaInfo could be mutated at any time. So you need to do all the check under the lock. 1795 // 1796 // Report back to NN bad block caused by non-existent block file. 1797 // WATCH-OUT: be sure the conditions checked above matches the following 1798 // method in FsDatasetImpl.java: 1799 // boolean isValidBlock(ExtendedBlock b) 1800 // all other conditions need to be true except that 1801 // replicaInfo.getBlockFile().exists() returns false . 1802 // I don't think we need the "WATCH-OUT" part. We shouldn't be calling isValidBlock , so why do we care if the check is the same as that check? I generally agree with this approach and I think we can get this in if that's fixed.
        Hide
        yzhangal Yongjun Zhang added a comment -

        HI Colin P. McCabe,

        Thanks for the review! I just uploaded rev 003 to address all the comments.

        BTW, about the WATCH-OUT, I was just thinking that someone could add another condition in the FsDatasetImpl#isValidBlock that makes the method to return false. But that's remote and probably won't happen.

        Thanks again.

        Show
        yzhangal Yongjun Zhang added a comment - HI Colin P. McCabe , Thanks for the review! I just uploaded rev 003 to address all the comments. BTW, about the WATCH-OUT, I was just thinking that someone could add another condition in the FsDatasetImpl#isValidBlock that makes the method to return false. But that's remote and probably won't happen. Thanks again.
        Hide
        cmccabe Colin P. McCabe added a comment -
        1786      boolean needToReportBadBlock = false;
        1787	      synchronized(data) {
        1788	        ReplicaInfo replicaInfo = (ReplicaInfo) data.getReplica(
        1789	            block.getBlockPoolId(), block.getBlockId());
        1790	        needToReportBadBlock = (replicaInfo != null
        1791	            && replicaInfo.getState() == ReplicaState.FINALIZED
        1792	            && !replicaInfo.getBlockFile().exists());
        1793	      }
        1794	      if (needToReportBadBlock)  {
        1795	        // Report back to NN bad block caused by non-existent block file.
        1796	        reportBadBlock(bpos, block, "Can't replicate block " + block
        1797	            + " because the block file doesn't exist");
        1798	      } else {
        1799	        String errStr = "Can't send invalid block " + block;
        1800	        LOG.info(errStr);
        1801	        bpos.trySendErrorReport(DatanodeProtocol.INVALID_BLOCK, errStr);
        1802	      }
        

        We shouldn't log a message saying that "the block file doesn't exist" if the block file exists, but is not finalized.

        I also don't see why we need to call FSDatasetSpi#getLength, if we already have access to the replica length here.

        I would suggest having your synchronized section set a string named replicaProblem. Then if the string is null at the end, there is no problem-- otherwise, the problem is contained in replicaProblem. Then you can check existence, replica state, and length all at once.

        BTW, about the WATCH-OUT, I was just thinking that someone could add another condition in the FsDatasetImpl#isValidBlock that makes the method to return false. But that's remote and probably won't happen.

        We don't even need to call isValidBlock. getReplica gives you all the info you need. Please take out this call, since it's unnecessary.

        Show
        cmccabe Colin P. McCabe added a comment - 1786 boolean needToReportBadBlock = false ; 1787 synchronized (data) { 1788 ReplicaInfo replicaInfo = (ReplicaInfo) data.getReplica( 1789 block.getBlockPoolId(), block.getBlockId()); 1790 needToReportBadBlock = (replicaInfo != null 1791 && replicaInfo.getState() == ReplicaState.FINALIZED 1792 && !replicaInfo.getBlockFile().exists()); 1793 } 1794 if (needToReportBadBlock) { 1795 // Report back to NN bad block caused by non-existent block file. 1796 reportBadBlock(bpos, block, "Can't replicate block " + block 1797 + " because the block file doesn't exist" ); 1798 } else { 1799 String errStr = "Can't send invalid block " + block; 1800 LOG.info(errStr); 1801 bpos.trySendErrorReport(DatanodeProtocol.INVALID_BLOCK, errStr); 1802 } We shouldn't log a message saying that "the block file doesn't exist" if the block file exists, but is not finalized. I also don't see why we need to call FSDatasetSpi#getLength , if we already have access to the replica length here. I would suggest having your synchronized section set a string named replicaProblem . Then if the string is null at the end, there is no problem-- otherwise, the problem is contained in replicaProblem . Then you can check existence, replica state, and length all at once. BTW, about the WATCH-OUT, I was just thinking that someone could add another condition in the FsDatasetImpl#isValidBlock that makes the method to return false. But that's remote and probably won't happen. We don't even need to call isValidBlock . getReplica gives you all the info you need. Please take out this call, since it's unnecessary.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12675964/HDFS-7235.003.patch
        against trunk revision e90718f.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        -1 javac. The applied patch generated 1304 javac compiler warnings (more than the trunk's current 1293 warnings).

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes
        org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing
        org.apache.hadoop.hdfs.server.namenode.ha.TestHAFsck
        org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits

        The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.fs.TestSymlinkHdfsFileSystem

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8462//testReport/
        Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8462//artifact/patchprocess/diffJavacWarnings.txt
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8462//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12675964/HDFS-7235.003.patch against trunk revision e90718f. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. -1 javac . The applied patch generated 1304 javac compiler warnings (more than the trunk's current 1293 warnings). +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing org.apache.hadoop.hdfs.server.namenode.ha.TestHAFsck org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.fs.TestSymlinkHdfsFileSystem +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8462//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8462//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8462//console This message is automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        HI Colin P. McCabe,

        Thanks again for the review. Please see my answer below.

        We shouldn't log a message saying that "the block file doesn't exist" if the block file exists, but is not finalized.

        We are not, we only log when the state is finalized, and the block file doesn't exist.

        I also don't see why we need to call FSDatasetSpi#getLength, if we already have access to the replica length here.

        The new fix we are introducing here is to handle a special case that when isValidBlock() returns false, so I tried to limit the change in the special handling block. If we remove the pre-exiisting FSDatasetSpi#getLength, we need to move the call getReplica() out of the false block.
        The getReplica() was marked @Deprecated, I consider calling it is a bit hack here already, Plus, we need to synchronize the whole block of code, so I hope we can limit the impact to within the false block. I wonder if this explanation makes sense to you.

        I would suggest having your synchronized section set a string named replicaProblem. Then if the string is null at the end, there is no problem-- otherwise, the problem is contained in replicaProblem. Then you can check existence, replica state, and length all at once.

        I am not sure I follow what you said, will check in person.

        We don't even need to call isValidBlock. getReplica gives you all the info you need. Please take out this call, since it's unnecessary.

        The {isValidBlock}} is an interface defined in FsDatasetSpi, and has its methods defined in derived classes FsDatasetImpl, and SimulatedFSDataset etc, which might have different implementation of the methods. It'd be nice to stick to the interface of FsDatasetSpi.

        Will discuss with you more.

        Thanks again.

        Show
        yzhangal Yongjun Zhang added a comment - HI Colin P. McCabe , Thanks again for the review. Please see my answer below. We shouldn't log a message saying that "the block file doesn't exist" if the block file exists, but is not finalized. We are not, we only log when the state is finalized, and the block file doesn't exist. I also don't see why we need to call FSDatasetSpi#getLength, if we already have access to the replica length here. The new fix we are introducing here is to handle a special case that when isValidBlock() returns false, so I tried to limit the change in the special handling block. If we remove the pre-exiisting FSDatasetSpi#getLength , we need to move the call getReplica() out of the false block. The getReplica() was marked @Deprecated , I consider calling it is a bit hack here already, Plus, we need to synchronize the whole block of code, so I hope we can limit the impact to within the false block. I wonder if this explanation makes sense to you. I would suggest having your synchronized section set a string named replicaProblem. Then if the string is null at the end, there is no problem-- otherwise, the problem is contained in replicaProblem. Then you can check existence, replica state, and length all at once. I am not sure I follow what you said, will check in person. We don't even need to call isValidBlock. getReplica gives you all the info you need. Please take out this call, since it's unnecessary. The {isValidBlock}} is an interface defined in FsDatasetSpi, and has its methods defined in derived classes FsDatasetImpl, and SimulatedFSDataset etc, which might have different implementation of the methods. It'd be nice to stick to the interface of FsDatasetSpi. Will discuss with you more. Thanks again.
        Hide
        cmccabe Colin P. McCabe added a comment -

        Hi Yongjun,

        Thanks for your patience here. I don't think the current patch is quite ready. I could point to a few things, like this: ReplicaInfo replicaInfo = (ReplicaInfo) data.getReplica( We shouldn't be downcasting here.

        I think the bigger issue is that the interface in FsDatasetSpi is just not very suitable to what we're trying to do. Rather than trying to hack it, I think we should come up with a better interface.

        I think we should replace FsDatasetSpi#isValid with this function:

          /**
           * Check if a block is valid.
           *
           * @param b           The block to check.
           * @param minLength   The minimum length that the block must have.  May be 0.
           * @param state       If this is null, it is ignored.  If it is non-null, we
           *                        will check that the replica has this state.
           *
           * @throws FileNotFoundException             If the replica is not found or there 
           *                                              was an error locating it.
           * @throws EOFException                      If the replica length is too short.
           * @throws UnexpectedReplicaStateException   If the replica is not in the 
           *                                             expected state.
           */
          public void checkBlock(ExtendedBlock b, long minLength, ReplicaState state);
        

        Since this function will throw a clearly marked exception detailing which case we're in, we won't have to call multiple functions. This will be better for performance since we're only taking the lock once. This will also be better for clarity, since the current APIs lead to some rather complex code.

        We could also get rid of FsDatasetSpi#isValidRbw, since this function can do everything that it can.
        Also UnexpectedReplicaStateException could be a new exception under hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java

        I think it's fine to change FsDatasetSpi for this (we did it when adding caching stuff, and again when adding "trash").

        Let me know what you think. I think it would make things a lot more clear.

        Show
        cmccabe Colin P. McCabe added a comment - Hi Yongjun, Thanks for your patience here. I don't think the current patch is quite ready. I could point to a few things, like this: ReplicaInfo replicaInfo = (ReplicaInfo) data.getReplica( We shouldn't be downcasting here. I think the bigger issue is that the interface in FsDatasetSpi is just not very suitable to what we're trying to do. Rather than trying to hack it, I think we should come up with a better interface. I think we should replace FsDatasetSpi#isValid with this function: /** * Check if a block is valid. * * @param b The block to check. * @param minLength The minimum length that the block must have. May be 0. * @param state If this is null , it is ignored. If it is non- null , we * will check that the replica has this state. * * @ throws FileNotFoundException If the replica is not found or there * was an error locating it. * @ throws EOFException If the replica length is too short . * @ throws UnexpectedReplicaStateException If the replica is not in the * expected state. */ public void checkBlock(ExtendedBlock b, long minLength, ReplicaState state); Since this function will throw a clearly marked exception detailing which case we're in, we won't have to call multiple functions. This will be better for performance since we're only taking the lock once. This will also be better for clarity, since the current APIs lead to some rather complex code. We could also get rid of FsDatasetSpi#isValidRbw , since this function can do everything that it can. Also UnexpectedReplicaStateException could be a new exception under hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java I think it's fine to change FsDatasetSpi for this (we did it when adding caching stuff, and again when adding "trash"). Let me know what you think. I think it would make things a lot more clear.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Colin P. McCabe, Thanks a lot for the side discussion and comment. I will look into.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Colin P. McCabe , Thanks a lot for the side discussion and comment. I will look into.
        Hide
        yzhangal Yongjun Zhang added a comment -

        HI Colin P. McCabe, I just uploaded a new rev (004) to address your comments. Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - HI Colin P. McCabe , I just uploaded a new rev (004) to address your comments. Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12676522/HDFS-7235.004.patch
        against trunk revision b94b8b3.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        -1 javac. The patch appears to cause the build to fail.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8491//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12676522/HDFS-7235.004.patch against trunk revision b94b8b3. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8491//console This message is automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Upload same patch 004 again since the test failure appears to irrelevant.

        Show
        yzhangal Yongjun Zhang added a comment - Upload same patch 004 again since the test failure appears to irrelevant.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12676534/HDFS-7235.004.patch
        against trunk revision d71d40a.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8494//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8494//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12676534/HDFS-7235.004.patch against trunk revision d71d40a. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestRollingUpgradeDowngrade +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8494//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8494//console This message is automatically generated.
        Hide
        cmccabe Colin P. McCabe added a comment -

        Thanks, Yongjun. This looks good overall.

        1792    try {
        1793	      data.checkBlock(block, block.getNumBytes(), ReplicaState.FINALIZED);
        1794	    }
        1795	    catch (ReplicaNotFoundException e) {
        1796	      replicaNotExist = true;
        1797	    }
        1798	    catch (UnexpectedReplicaStateException e) {
        1799	      replicaStateNotFinalized = true;
        1800	    }
        1801	    catch (FileNotFoundException e) {
        1802	      blockFileNotExist = true;
        1803	    }
        1804	    catch (EOFException e) {
        1805	      lengthTooShort = true;
        1806	    }
        

        Nit: can you put the catches on the same line as the previous end bracket? The Oracle Java style guide says to do this (section 7.9). It saves vertical space. Same comment in FsDatasetImpl#isValidRbw and SimulatedFSDataset.java.

        27     /**
        28	 * Exception indicating that DataNode does not have a replica
        29	 * that matches the target block.  
        30	 */
        31	public class UnexpectedReplicaStateException extends IOException {
        

        Hmm... shouldn't this be "Exception indicating that the replica is in an unexpected state"? "Does not have a replica" is a different error case.

        There are a bunch of unnecessary whitespace changes in this patch... for example, in FsDatasetImpl.java. It's best to avoid those because they can confuse people looking through the history. I like to use "meld" to let me see and quickly eliminate spurious whitespace changes in my patch.

        Looks good aside from that!

        Show
        cmccabe Colin P. McCabe added a comment - Thanks, Yongjun. This looks good overall. 1792 try { 1793 data.checkBlock(block, block.getNumBytes(), ReplicaState.FINALIZED); 1794 } 1795 catch (ReplicaNotFoundException e) { 1796 replicaNotExist = true ; 1797 } 1798 catch (UnexpectedReplicaStateException e) { 1799 replicaStateNotFinalized = true ; 1800 } 1801 catch (FileNotFoundException e) { 1802 blockFileNotExist = true ; 1803 } 1804 catch (EOFException e) { 1805 lengthTooShort = true ; 1806 } Nit: can you put the catches on the same line as the previous end bracket? The Oracle Java style guide says to do this (section 7.9). It saves vertical space. Same comment in FsDatasetImpl#isValidRbw and SimulatedFSDataset.java . 27 /** 28 * Exception indicating that DataNode does not have a replica 29 * that matches the target block. 30 */ 31 public class UnexpectedReplicaStateException extends IOException { Hmm... shouldn't this be "Exception indicating that the replica is in an unexpected state"? "Does not have a replica" is a different error case. There are a bunch of unnecessary whitespace changes in this patch... for example, in FsDatasetImpl.java. It's best to avoid those because they can confuse people looking through the history. I like to use "meld" to let me see and quickly eliminate spurious whitespace changes in my patch. Looks good aside from that!
        Hide
        yzhangal Yongjun Zhang added a comment -

        Many thanks Colin, just uploaded rev 005 to address all your comments.

        Show
        yzhangal Yongjun Zhang added a comment - Many thanks Colin, just uploaded rev 005 to address all your comments.
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12676833/HDFS-7235.005.patch
        against trunk revision 57dec28.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8516//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8516//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12676833/HDFS-7235.005.patch against trunk revision 57dec28. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8516//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8516//console This message is automatically generated.
        Hide
        cmccabe Colin P. McCabe added a comment -
        1791	    try {
        1792	      data.checkBlock(block, block.getNumBytes(), ReplicaState.FINALIZED);
        1793	    } catch (ReplicaNotFoundException e) {
        1794	      replicaNotExist = true;
        1795	    } catch (UnexpectedReplicaStateException e) {
        1796	      replicaStateNotFinalized = true;
        1797	    } catch (FileNotFoundException e) {
        1798	      blockFileNotExist = true;
        1799	    } catch (EOFException e) {
        1800	      lengthTooShort = true;
        1801	    }
        

        We should also add a catch block for IOException, which may occur while we're getting the block file length. Probably this should be handled as blockFileNotExist, since the block file was inaccessible.

        +1 once this is addressed

        Show
        cmccabe Colin P. McCabe added a comment - 1791 try { 1792 data.checkBlock(block, block.getNumBytes(), ReplicaState.FINALIZED); 1793 } catch (ReplicaNotFoundException e) { 1794 replicaNotExist = true ; 1795 } catch (UnexpectedReplicaStateException e) { 1796 replicaStateNotFinalized = true ; 1797 } catch (FileNotFoundException e) { 1798 blockFileNotExist = true ; 1799 } catch (EOFException e) { 1800 lengthTooShort = true ; 1801 } We should also add a catch block for IOException, which may occur while we're getting the block file length. Probably this should be handled as blockFileNotExist , since the block file was inaccessible. +1 once this is addressed
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Colin,

        Good catch of yours. When getLength() throws IOException, the pre-existing behaviour was to issue a warn message from the caller side of DataNode#transferBlock, so I was trying to keep that behaviour. But after discussing with you, I agree that we should report bad block for this scenario because the block file was not accessible. I just uploaded rev 006 to incorporate this change. Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Colin, Good catch of yours. When getLength() throws IOException, the pre-existing behaviour was to issue a warn message from the caller side of DataNode#transferBlock , so I was trying to keep that behaviour. But after discussing with you, I agree that we should report bad block for this scenario because the block file was not accessible. I just uploaded rev 006 to incorporate this change. Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12677035/HDFS-7235.006.patch
        against trunk revision 683897f.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The test build failed in hadoop-hdfs-project/hadoop-hdfs

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8536//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8536//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12677035/HDFS-7235.006.patch against trunk revision 683897f. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The test build failed in hadoop-hdfs-project/hadoop-hdfs +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8536//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8536//console This message is automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        The test result shows no failure, except the log has the following in the end:

        + mv /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build@2/../patchprocess /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build@2/patchprocess
        mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build@2/../patchprocess': No such file or directory
        Build step 'Execute shell' marked build as failure
        Archiving artifacts
        Description set: HDFS-7235
        Recording test results
        Publish JUnit test result report is waiting for a checkpoint on PreCommit-HDFS-Build #8535
        Finished: FAILURE
        

        which happens to other tests from time to time. It appears to be a test infrastructure issue.

        Show
        yzhangal Yongjun Zhang added a comment - The test result shows no failure, except the log has the following in the end: + mv /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build@2/../patchprocess /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build@2/patchprocess mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build@2/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Description set: HDFS-7235 Recording test results Publish JUnit test result report is waiting for a checkpoint on PreCommit-HDFS-Build #8535 Finished: FAILURE which happens to other tests from time to time. It appears to be a test infrastructure issue.
        Hide
        cmccabe Colin P. McCabe added a comment -

        Thanks, Yongjun. +1.

        I will re-trigger jenkins to see if we can get a clean(er) build.

        Show
        cmccabe Colin P. McCabe added a comment - Thanks, Yongjun. +1. I will re-trigger jenkins to see if we can get a clean(er) build.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12677035/HDFS-7235.006.patch
        against trunk revision ce1a441.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8551//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12677035/HDFS-7235.006.patch against trunk revision ce1a441. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8551//console This message is automatically generated.
        Hide
        cmccabe Colin P. McCabe added a comment -

        Yongjun, the patch doesn't apply against trunk any more. Can you rebase? thanks

        Show
        cmccabe Colin P. McCabe added a comment - Yongjun, the patch doesn't apply against trunk any more. Can you rebase? thanks
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Colin, just uploaded a rebased patch.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Colin, just uploaded a rebased patch.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12677434/HDFS-7235.007.patch
        against trunk revision a16d022.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell:

        org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8557//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8557//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12677434/HDFS-7235.007.patch against trunk revision a16d022. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell: org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8557//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8557//console This message is automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12677534/HDFS-7235.007.patch
        against trunk revision 971e91c.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8562//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12677534/HDFS-7235.007.patch against trunk revision 971e91c. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8562//console This message is automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        There was TestDistributedShell failure reported as YARN-2607, the symptom there looks a bit different than the one reported above. I ran the same test locally before (trunk tip 5b1dfe78b8b06335bed0bcb83f12bb936d4c021b) and after the patc, they failed the same way, but the symptom running locally is different than YARN-2607, and the report above.

        Seems this test need some more study, just uploaded the same patch here again to see how it runs.

        Show
        yzhangal Yongjun Zhang added a comment - There was TestDistributedShell failure reported as YARN-2607 , the symptom there looks a bit different than the one reported above. I ran the same test locally before (trunk tip 5b1dfe78b8b06335bed0bcb83f12bb936d4c021b) and after the patc, they failed the same way, but the symptom running locally is different than YARN-2607 , and the report above. Seems this test need some more study, just uploaded the same patch here again to see how it runs.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12677536/HDFS-7235.007.patch
        against trunk revision 971e91c.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8564//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12677536/HDFS-7235.007.patch against trunk revision 971e91c. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8564//console This message is automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        I was able to apply the patch locally even at the latest tip of trunk

        commit 58c0bb9ed9f4a2491395b63c68046562a73526c9
        Author: yliu <yliu@apache.org>
        Date: Tue Oct 28 21:11:31 2014 +0800

        Show
        yzhangal Yongjun Zhang added a comment - I was able to apply the patch locally even at the latest tip of trunk commit 58c0bb9ed9f4a2491395b63c68046562a73526c9 Author: yliu <yliu@apache.org> Date: Tue Oct 28 21:11:31 2014 +0800
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12677607/HDFS-7235.007.patch
        against trunk revision 58c0bb9.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8569//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8569//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12677607/HDFS-7235.007.patch against trunk revision 58c0bb9. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8569//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8569//console This message is automatically generated.
        Hide
        cmccabe Colin P. McCabe added a comment -

        Committed. Thanks, Yongjun.

        Show
        cmccabe Colin P. McCabe added a comment - Committed. Thanks, Yongjun.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Many thanks Colin!

        Show
        yzhangal Yongjun Zhang added a comment - Many thanks Colin!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #6375 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6375/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (cmccabe: rev ac9ab037e9a9b03e4fa9bd471d3ab9940beb53fb)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #6375 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6375/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (cmccabe: rev ac9ab037e9a9b03e4fa9bd471d3ab9940beb53fb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #727 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/727/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (cmccabe: rev ac9ab037e9a9b03e4fa9bd471d3ab9940beb53fb)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #727 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/727/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (cmccabe: rev ac9ab037e9a9b03e4fa9bd471d3ab9940beb53fb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #1941 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1941/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (cmccabe: rev ac9ab037e9a9b03e4fa9bd471d3ab9940beb53fb)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1941 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1941/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (cmccabe: rev ac9ab037e9a9b03e4fa9bd471d3ab9940beb53fb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #1916 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1916/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (cmccabe: rev ac9ab037e9a9b03e4fa9bd471d3ab9940beb53fb)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1916 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1916/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (cmccabe: rev ac9ab037e9a9b03e4fa9bd471d3ab9940beb53fb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/UnexpectedReplicaStateException.java
        Hide
        vinayrpet Vinayakumar B added a comment -

        Cherry-picked to 2.6.1

        Show
        vinayrpet Vinayakumar B added a comment - Cherry-picked to 2.6.1
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8298 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8298/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8298 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8298/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #287 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/287/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #287 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/287/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1017 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1017/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1017 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1017/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #284 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/284/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #284 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/284/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2233 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2233/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2233 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2233/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2214 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2214/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2214 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2214/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #276 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/276/)
        HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #276 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/276/ ) HDFS-7235 . DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) (vinayakumarb: rev f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        This wasn't originally in 2.6.1, must have been committed to 2.6, which was already 2.6.2. I just committed this to 2.6.1 taking Sangjin Lee cherry-pick, which must have come from branch-2.6.

        Ran compilation and TestReplication before the push.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - This wasn't originally in 2.6.1, must have been committed to 2.6, which was already 2.6.2. I just committed this to 2.6.1 taking Sangjin Lee cherry-pick, which must have come from branch-2.6. Ran compilation and TestReplication before the push.

          People

          • Assignee:
            yzhangal Yongjun Zhang
            Reporter:
            yzhangal Yongjun Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development