Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10625

VolumeScanner to report why a block is found bad

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.7.4
    • Component/s: datanode, hdfs
    • Labels:
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      VolumeScanner may report:

      WARN org.apache.hadoop.hdfs.server.datanode.VolumeScanner: Reporting bad blk_1170125248_96458336 on /d/dfs/dn
      

      It would be helpful to report the reason why the block is bad, especially when the block is corrupt, where is the first corrupted chunk in the block.

      1. HDFS-10625.003.patch
        3 kB
        Yiqun Lin
      2. HDFS-10625.004.patch
        3 kB
        Yiqun Lin
      3. HDFS-10625.005.patch
        3 kB
        Rushabh S Shah
      4. HDFS-10625.patch
        0.8 kB
        Rushabh S Shah
      5. HDFS-10625-1.patch
        2 kB
        Rushabh S Shah

        Issue Links

          Activity

          Hide
          shahrs87 Rushabh S Shah added a comment -

          Please review.

          Show
          shahrs87 Rushabh S Shah added a comment - Please review.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 32s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 40s trunk passed
          +1 compile 1m 1s trunk passed
          +1 checkstyle 0m 28s trunk passed
          +1 mvnsite 1m 10s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 59s trunk passed
          +1 javadoc 0m 55s trunk passed
          +1 mvninstall 0m 56s the patch passed
          +1 compile 0m 54s the patch passed
          +1 javac 0m 54s the patch passed
          +1 checkstyle 0m 23s the patch passed
          +1 mvnsite 1m 5s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 10s the patch passed
          +1 javadoc 1m 0s the patch passed
          +1 unit 74m 7s hadoop-hdfs in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          96m 43s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817952/HDFS-10625.patch
          JIRA Issue HDFS-10625
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 8fc773df0d4f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 54bf14f
          Default Java 1.8.0_91
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16057/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16057/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 32s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 40s trunk passed +1 compile 1m 1s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 1m 10s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 59s trunk passed +1 javadoc 0m 55s trunk passed +1 mvninstall 0m 56s the patch passed +1 compile 0m 54s the patch passed +1 javac 0m 54s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 1m 5s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 10s the patch passed +1 javadoc 1m 0s the patch passed +1 unit 74m 7s hadoop-hdfs in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 96m 43s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817952/HDFS-10625.patch JIRA Issue HDFS-10625 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8fc773df0d4f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 54bf14f Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16057/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16057/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          looks good to me.
          VolumeScanner uses BlockSender and a null stream to verify the integrity of the block. If the block is corrupt, BlockSender#verifyChecksum throws a ChecksumException, which details the location of the corruption.

          throw new ChecksumException("Checksum failed at " + failedPos,
                      failedPos);
          
          Show
          jojochuang Wei-Chiu Chuang added a comment - looks good to me. VolumeScanner uses BlockSender and a null stream to verify the integrity of the block. If the block is corrupt, BlockSender#verifyChecksum throws a ChecksumException, which details the location of the corruption. throw new ChecksumException( "Checksum failed at " + failedPos, failedPos);
          Hide
          yzhangal Yongjun Zhang added a comment - - edited

          Thanks Rushabh S Shah for the patch and Wei-Chiu Chuang for the comment.

          It'd be nice to include the length of the replica, visible length, ondisk length etc in the report too. Suggest to use the same method used here (see HDFS-10587),

          016-04-15 22:03:05,066 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Recovering ReplicaBeingWritten, blk_1556997324_1100153495099, RBW
          getNumBytes() = 41381376
          getBytesOnDisk() = 41381376
          getVisibleLength()= 41186444
          getVolume() = /hadoop-i/data/current
          getBlockFile() = /hadoop-i/data/current/BP-1043567091-10.1.1.1-1343682168507/current/rbw/blk_1556997324
          bytesAcked=41186444
          bytesOnDisk=41381376
          
          Show
          yzhangal Yongjun Zhang added a comment - - edited Thanks Rushabh S Shah for the patch and Wei-Chiu Chuang for the comment. It'd be nice to include the length of the replica, visible length, ondisk length etc in the report too. Suggest to use the same method used here (see HDFS-10587 ), 016-04-15 22:03:05,066 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Recovering ReplicaBeingWritten, blk_1556997324_1100153495099, RBW getNumBytes() = 41381376 getBytesOnDisk() = 41381376 getVisibleLength()= 41186444 getVolume() = /hadoop-i/data/current getBlockFile() = /hadoop-i/data/current/BP-1043567091-10.1.1.1-1343682168507/current/rbw/blk_1556997324 bytesAcked=41186444 bytesOnDisk=41381376
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Thanks Yongjun Zhang for the review.
          Added a new patch to address your comments.

          I ran TestBlockScanner#testCorruptBlockHandling to confirm the exception message:

          2016-07-15 13:50:35,445 [VolumeScannerThread(/Users/rushabhs/hadoop/Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data1)] INFO  datanode.TestBlockScanner (TestBlockScanner.java:handle(325)) - handling block BP-449958863-xxxxx-1468608632636:blk_1073741828_1004 (exception org.apache.hadoop.fs.ChecksumException: Checksum failed at 0 for replica: 
          FinalizedReplica, blk_1073741828_1004, FINALIZED
            getNumBytes()     = 4
            getBytesOnDisk()  = 4
            getVisibleLength()= 4
            getVolume()       = /Users/rushabhs/hadoop/Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data1/current
            getBlockFile()    = /Users/rushabhs/hadoop/Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data1/current/BP-449958863-xxxxx-1468608632636/current/finalized/subdir0/subdir0/blk_1073741828)
          

          Please review the revised patch.

          Show
          shahrs87 Rushabh S Shah added a comment - Thanks Yongjun Zhang for the review. Added a new patch to address your comments. I ran TestBlockScanner#testCorruptBlockHandling to confirm the exception message: 2016-07-15 13:50:35,445 [VolumeScannerThread(/Users/rushabhs/hadoop/Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data1)] INFO datanode.TestBlockScanner (TestBlockScanner.java:handle(325)) - handling block BP-449958863-xxxxx-1468608632636:blk_1073741828_1004 (exception org.apache.hadoop.fs.ChecksumException: Checksum failed at 0 for replica: FinalizedReplica, blk_1073741828_1004, FINALIZED getNumBytes() = 4 getBytesOnDisk() = 4 getVisibleLength()= 4 getVolume() = /Users/rushabhs/hadoop/Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data1/current getBlockFile() = /Users/rushabhs/hadoop/Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data1/current/BP-449958863-xxxxx-1468608632636/current/finalized/subdir0/subdir0/blk_1073741828) Please review the revised patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 16s trunk passed
          +1 compile 0m 48s trunk passed
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 47s trunk passed
          +1 javadoc 0m 56s trunk passed
          +1 mvninstall 0m 51s the patch passed
          +1 compile 0m 45s the patch passed
          +1 javac 0m 45s the patch passed
          +1 checkstyle 0m 24s the patch passed
          +1 mvnsite 0m 53s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 55s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 0m 54s the patch passed
          +1 unit 59m 3s hadoop-hdfs in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          79m 11s



          Reason Tests
          FindBugs module:hadoop-hdfs-project/hadoop-hdfs
            org.apache.hadoop.hdfs.server.datanode.BlockSender.verifyChecksum(byte[], int, int, int, int) might ignore org.apache.hadoop.hdfs.server.datanode.ReplicaNotFoundException At BlockSender.java:might ignore org.apache.hadoop.hdfs.server.datanode.ReplicaNotFoundException At BlockSender.java:[line 693]



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818217/HDFS-10625-1.patch
          JIRA Issue HDFS-10625
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux a663552a7059 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c48e9d6
          Default Java 1.8.0_91
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/16069/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16069/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16069/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 16s trunk passed +1 compile 0m 48s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 51s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 55s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 0m 54s the patch passed +1 unit 59m 3s hadoop-hdfs in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 79m 11s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   org.apache.hadoop.hdfs.server.datanode.BlockSender.verifyChecksum(byte[], int, int, int, int) might ignore org.apache.hadoop.hdfs.server.datanode.ReplicaNotFoundException At BlockSender.java:might ignore org.apache.hadoop.hdfs.server.datanode.ReplicaNotFoundException At BlockSender.java: [line 693] Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818217/HDFS-10625-1.patch JIRA Issue HDFS-10625 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a663552a7059 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c48e9d6 Default Java 1.8.0_91 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/16069/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16069/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16069/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Yongjun Zhang Wei-Chiu Chuang: any further comments on the latest patch ?

          Show
          shahrs87 Rushabh S Shah added a comment - Yongjun Zhang Wei-Chiu Chuang : any further comments on the latest patch ?
          Hide
          linyiqun Yiqun Lin added a comment -

          Hi, Rushabh S Shah, this jira is similar to HDFS-10626. Some codes are both changed in these two jira. According to the latest comment from Yongjun Zhang in HDFS-10626, I have also changed the code from:

          LOG.warn("Reporting bad {} on {}", block, volume.getBasePath());
          

          to

          LOG.warn("Reporting bad " + block + " with volume "
                    + volume.getBasePath(), e);
          

          I'm not sure if this line should be also changed in this jira? Yongjun Zhang, can take a look for this, and make a coordination.

          Show
          linyiqun Yiqun Lin added a comment - Hi, Rushabh S Shah , this jira is similar to HDFS-10626 . Some codes are both changed in these two jira. According to the latest comment from Yongjun Zhang in HDFS-10626 , I have also changed the code from: LOG.warn( "Reporting bad {} on {}" , block, volume.getBasePath()); to LOG.warn( "Reporting bad " + block + " with volume " + volume.getBasePath(), e); I'm not sure if this line should be also changed in this jira? Yongjun Zhang , can take a look for this, and make a coordination.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks Yiqun Lin for noticing that. sorry for being late on this, I will look into early next week.

          Show
          yzhangal Yongjun Zhang added a comment - Thanks Yiqun Lin for noticing that. sorry for being late on this, I will look into early next week.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Yiqun Lin and Rushabh S Shah,

          Sorry for the delay. I took a further look, and think it's good to include the HDFS-10626 fix here, and mark HDFS-10626 as a duplicate. I'd like to include both of you as contributers for this jira.

          I looked at the latest patch here, it looks to me that the best place to fix is in BlockSender

            long sendBlock(DataOutputStream out, OutputStream baseStream, 
                           DataTransferThrottler throttler) throws IOException {
              final TraceScope scope = datanode.getTracer().
                  newScope("sendBlock_" + block.getBlockId());
              try {
                return doSendBlock(out, baseStream, throttler);
              } finally {
                scope.close();
              }
            }
          

          We can add a catch block here to catch the IOException thrown, then include the replica information and throw a new IO exception, e.g:

              try {
                return doSendBlock(out, baseStream, throttler);
              } catch (IOException ie) {
                // throw new IOE here with replica info
                throw new IOException(replicaInfoStr, ie);
              } finally {
                scope.close();
              }
          

          There is a snippet in the constructor to get the replica info:

               final Replica replica;
                final long replicaVisibleLength;
                synchronized(datanode.data) { 
                  replica = getReplica(block, datanode);
                  replicaVisibleLength = replica.getVisibleLength();
                }
          

          Looks like we can make this replica a member of BlockSender instead of a local variable here, so that we can refer to it when needed, such as for this jira. We probably should make replicaVisibleLength a member and report it as part of the replica info too, since when the writing is going on, this value may be changing concurrently. Hi Vinayakumar B, what do you think about this suggestion?

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Hi Yiqun Lin and Rushabh S Shah , Sorry for the delay. I took a further look, and think it's good to include the HDFS-10626 fix here, and mark HDFS-10626 as a duplicate. I'd like to include both of you as contributers for this jira. I looked at the latest patch here, it looks to me that the best place to fix is in BlockSender long sendBlock(DataOutputStream out, OutputStream baseStream, DataTransferThrottler throttler) throws IOException { final TraceScope scope = datanode.getTracer(). newScope( "sendBlock_" + block.getBlockId()); try { return doSendBlock(out, baseStream, throttler); } finally { scope.close(); } } We can add a catch block here to catch the IOException thrown, then include the replica information and throw a new IO exception, e.g: try { return doSendBlock(out, baseStream, throttler); } catch (IOException ie) { // throw new IOE here with replica info throw new IOException(replicaInfoStr, ie); } finally { scope.close(); } There is a snippet in the constructor to get the replica info: final Replica replica; final long replicaVisibleLength; synchronized (datanode.data) { replica = getReplica(block, datanode); replicaVisibleLength = replica.getVisibleLength(); } Looks like we can make this replica a member of BlockSender instead of a local variable here, so that we can refer to it when needed, such as for this jira. We probably should make replicaVisibleLength a member and report it as part of the replica info too, since when the writing is going on, this value may be changing concurrently. Hi Vinayakumar B , what do you think about this suggestion? Thanks.
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks Yongjun Zhang for the comment. I will post a new patch for this after Vinayakumar B's feedback.

          Show
          linyiqun Yiqun Lin added a comment - Thanks Yongjun Zhang for the comment. I will post a new patch for this after Vinayakumar B 's feedback.
          Hide
          vinayrpet Vinayakumar B added a comment -

          We can add a catch block here to catch the IOException thrown, then include the replica information and throw a new IO exception, e.g:

          One problem here, is for the places which expects Specific exception such as ChecksumException or FileNotFoundException, they get IOException with cause set as ChecksumException or FNFE.
          So its better to not to change in this. Let original IOException thrown back. Anyway DN logs will be there to catch the replica details.

          Looks like we can make this replica a member of BlockSender instead of a local variable here, so that we can refer to it when needed, such as for this jira. We probably should make replicaVisibleLength a member and report it as part of the replica info too, since when the writing is going on, this value may be changing concurrently.

          Making ReplicaInfo a member is good, but making replicaVisibleLength a member may not be required. Because already endOffSet will be present which can decide how much BlockSender intended to read. So whenever required endOffset can be used.
          Coming to checksum verfication, BlockSender will do checkSum verification for only finalized blocks via VolumeScanner. Not while reading(Reading case verification happens at the client). So we can expect replica can be finalized in this case and no change in the visibleLength.

          So I feel, for the latest patch change required is, combining HDFS-10626, making replicaInfo a member and using to construct checksumException message.

          Show
          vinayrpet Vinayakumar B added a comment - We can add a catch block here to catch the IOException thrown, then include the replica information and throw a new IO exception, e.g: One problem here, is for the places which expects Specific exception such as ChecksumException or FileNotFoundException , they get IOException with cause set as ChecksumException or FNFE. So its better to not to change in this. Let original IOException thrown back. Anyway DN logs will be there to catch the replica details. Looks like we can make this replica a member of BlockSender instead of a local variable here, so that we can refer to it when needed, such as for this jira. We probably should make replicaVisibleLength a member and report it as part of the replica info too, since when the writing is going on, this value may be changing concurrently. Making ReplicaInfo a member is good, but making replicaVisibleLength a member may not be required. Because already endOffSet will be present which can decide how much BlockSender intended to read. So whenever required endOffset can be used. Coming to checksum verfication, BlockSender will do checkSum verification for only finalized blocks via VolumeScanner. Not while reading(Reading case verification happens at the client). So we can expect replica can be finalized in this case and no change in the visibleLength. So I feel, for the latest patch change required is, combining HDFS-10626 , making replicaInfo a member and using to construct checksumException message.
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks a lot for the comment, Vinayakumar B.

          One problem here, is for the places which expects Specific exception such as ChecksumException or FileNotFoundException, they get IOException with cause set as ChecksumException or FNFE.
          So its better to not to change in this. Let original IOException thrown back. Anyway DN logs will be there to catch the replica details.

          One comment from me for this: Can we print the log replica info when catch the IOException? So that we can not only see the replica info and also keep the original IOException.

          Other suggestions look good to me. Post a new patch for addressing the latest comment, thanks for review.

          Show
          linyiqun Yiqun Lin added a comment - Thanks a lot for the comment, Vinayakumar B . One problem here, is for the places which expects Specific exception such as ChecksumException or FileNotFoundException, they get IOException with cause set as ChecksumException or FNFE. So its better to not to change in this. Let original IOException thrown back. Anyway DN logs will be there to catch the replica details. One comment from me for this: Can we print the log replica info when catch the IOException? So that we can not only see the replica info and also keep the original IOException. Other suggestions look good to me. Post a new patch for addressing the latest comment, thanks for review.
          Hide
          vinayrpet Vinayakumar B added a comment -

          One comment from me for this: Can we print the log replica info when catch the IOException? So that we can not only see the replica info and also keep the original IOException.

          I see that all current cases of sendBlock() can print the exception messages with blockId down the line. So IMO we can avoid multiple redundant logs about same Exception. Already similar cases are there which would flood the logs by logging redundant logs in multiple levels. Example, any exception during the writing, will log at multiple places.
          Better we can avoid redundant logging.

          Show
          vinayrpet Vinayakumar B added a comment - One comment from me for this: Can we print the log replica info when catch the IOException? So that we can not only see the replica info and also keep the original IOException. I see that all current cases of sendBlock() can print the exception messages with blockId down the line. So IMO we can avoid multiple redundant logs about same Exception. Already similar cases are there which would flood the logs by logging redundant logs in multiple levels. Example, any exception during the writing, will log at multiple places. Better we can avoid redundant logging.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 55s trunk passed
          +1 compile 0m 52s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 3s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 52s trunk passed
          +1 javadoc 1m 4s trunk passed
          +1 mvninstall 0m 58s the patch passed
          +1 compile 0m 45s the patch passed
          +1 javac 0m 45s the patch passed
          -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 58 unchanged - 1 fixed = 60 total (was 59)
          +1 mvnsite 0m 58s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 56s the patch passed
          +1 javadoc 0m 53s the patch passed
          -1 unit 70m 34s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          90m 54s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.cli.TestHDFSCLI
          Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820659/HDFS-10625.003.patch
          JIRA Issue HDFS-10625
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3b9fc1f122f8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8d06bda
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16232/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16232/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16232/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16232/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 55s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 3s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 52s trunk passed +1 javadoc 1m 4s trunk passed +1 mvninstall 0m 58s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 58 unchanged - 1 fixed = 60 total (was 59) +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 56s the patch passed +1 javadoc 0m 53s the patch passed -1 unit 70m 34s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 90m 54s Reason Tests Failed junit tests hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.namenode.TestEditLog   hadoop.cli.TestHDFSCLI Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820659/HDFS-10625.003.patch JIRA Issue HDFS-10625 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3b9fc1f122f8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8d06bda Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16232/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16232/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16232/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16232/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks Vinayakumar B for the review and Yiqun Lin for the new rev.

          The patch looks good to me. Does it look good to you Vinay? The unit test failures look unrelated. I know org.apache.hadoop.cli.TestHDFSCLI.testAll is solved by HDFS-10696. Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Thanks Vinayakumar B for the review and Yiqun Lin for the new rev. The patch looks good to me. Does it look good to you Vinay? The unit test failures look unrelated. I know org.apache.hadoop.cli.TestHDFSCLI.testAll is solved by HDFS-10696 . Thanks.
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Thanks Yiqun Lin for updating the patch!
          Thanks Vinayakumar B and Yongjun Zhang for the reviews !
          The latest patch looks good to me.
          +1 (non-binding)

          Show
          shahrs87 Rushabh S Shah added a comment - Thanks Yiqun Lin for updating the patch! Thanks Vinayakumar B and Yongjun Zhang for the reviews ! The latest patch looks good to me. +1 (non-binding)
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks Yongjun Zhang and Rushabh S Shah for the review.
          Hi, Vinayakumar B, does the latest patch also looks good to you? If yes, I think we can commit this to trunk.

          Show
          linyiqun Yiqun Lin added a comment - Thanks Yongjun Zhang and Rushabh S Shah for the review. Hi, Vinayakumar B , does the latest patch also looks good to you? If yes, I think we can commit this to trunk.
          Hide
          vinayrpet Vinayakumar B added a comment -

          one nit, I thought it was already mentioned.
          replicaInfoString.append(" for replica: \n" + replica.toString());
          "\n" can be avoided here, as it helps to grep logs with complete message.

          +1 once addressed.

          Show
          vinayrpet Vinayakumar B added a comment - one nit, I thought it was already mentioned. replicaInfoString.append(" for replica: \n" + replica.toString()); "\n" can be avoided here, as it helps to grep logs with complete message. +1 once addressed.
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks a lot for the review, Vinayakumar B. Post a new patch for addressing that. I believe this patch will be the final patch of this jira, .

          Show
          linyiqun Yiqun Lin added a comment - Thanks a lot for the review, Vinayakumar B . Post a new patch for addressing that. I believe this patch will be the final patch of this jira, .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 31s trunk passed
          +1 compile 0m 46s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 59s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 2m 1s trunk passed
          +1 javadoc 1m 0s trunk passed
          +1 mvninstall 1m 4s the patch passed
          +1 compile 0m 58s the patch passed
          +1 javac 0m 58s the patch passed
          -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 58 unchanged - 1 fixed = 60 total (was 59)
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 57s the patch passed
          +1 javadoc 0m 54s the patch passed
          -1 unit 78m 16s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          100m 7s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821820/HDFS-10625.004.patch
          JIRA Issue HDFS-10625
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux fb852dea8bdd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d848184
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16307/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16307/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16307/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16307/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 31s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 1s trunk passed +1 javadoc 1m 0s trunk passed +1 mvninstall 1m 4s the patch passed +1 compile 0m 58s the patch passed +1 javac 0m 58s the patch passed -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 58 unchanged - 1 fixed = 60 total (was 59) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 57s the patch passed +1 javadoc 0m 54s the patch passed -1 unit 78m 16s hadoop-hdfs in the patch failed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 100m 7s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821820/HDFS-10625.004.patch JIRA Issue HDFS-10625 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fb852dea8bdd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d848184 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16307/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16307/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16307/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16307/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          shahrs87 Rushabh S Shah added a comment - - edited

          Vinayakumar B: Can you please review the latest patch (005 version).
          I think this should be the last pass.

          Show
          shahrs87 Rushabh S Shah added a comment - - edited Vinayakumar B : Can you please review the latest patch (005 version). I think this should be the last pass.
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Cancelling the patch to address the checkstyle issues.

          Show
          shahrs87 Rushabh S Shah added a comment - Cancelling the patch to address the checkstyle issues.
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Attaching a new patch.
          Just addressed the checkstyle warning.

           diff ~/patches/jira/HDFS-10625.00*
          2c2
          < index 398935d..902d28b 100644
          ---
          > index 398935d..e4ebe82 100644
          9c9
          < +  /** The replica of the block that is being read */
          ---
          > +  /** The replica of the block that is being read. */
          
          Show
          shahrs87 Rushabh S Shah added a comment - Attaching a new patch. Just addressed the checkstyle warning. diff ~/patches/jira/HDFS-10625.00* 2c2 < index 398935d..902d28b 100644 --- > index 398935d..e4ebe82 100644 9c9 < + /** The replica of the block that is being read */ --- > + /** The replica of the block that is being read. */
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 48s trunk passed
          +1 compile 0m 46s trunk passed
          +1 checkstyle 0m 25s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 39s trunk passed
          +1 javadoc 0m 56s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 41s the patch passed
          +1 javac 0m 41s the patch passed
          -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 58 unchanged - 1 fixed = 59 total (was 59)
          +1 mvnsite 0m 50s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 46s the patch passed
          +1 javadoc 0m 52s the patch passed
          -1 unit 66m 30s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          86m 16s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823122/HDFS-10625.005.patch
          JIRA Issue HDFS-10625
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 42982d8bbc18 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 89c0bff
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16391/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16391/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16391/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 48s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 39s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 41s the patch passed +1 javac 0m 41s the patch passed -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 58 unchanged - 1 fixed = 59 total (was 59) +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 52s the patch passed -1 unit 66m 30s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 86m 16s Reason Tests Failed junit tests hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823122/HDFS-10625.005.patch JIRA Issue HDFS-10625 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 42982d8bbc18 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 89c0bff Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16391/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16391/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16391/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          +1 for v5 patch.
          checkstyle for method length can be ignored.

          Show
          vinayrpet Vinayakumar B added a comment - +1 for v5 patch. checkstyle for method length can be ignored.
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Both the failed tests: hadoop.hdfs.server.balancer.TestBalancer and hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation are unrelated to this patch.
          Both are passing fine on my machine.

          Show
          shahrs87 Rushabh S Shah added a comment - Both the failed tests: hadoop.hdfs.server.balancer.TestBalancer and hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation are unrelated to this patch. Both are passing fine on my machine.
          Hide
          shahrs87 Rushabh S Shah added a comment - - edited

          Yongjun Zhang, Vinayakumar B: can someone please commit the latest patch ?

          Show
          shahrs87 Rushabh S Shah added a comment - - edited Yongjun Zhang , Vinayakumar B : can someone please commit the latest patch ?
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks Rushabh S Shah for fixing checkstyle issues. I have closed HDFS-10626. Anyone who can help commit the latest patch? Thanks in advance.

          Show
          linyiqun Yiqun Lin added a comment - Thanks Rushabh S Shah for fixing checkstyle issues. I have closed HDFS-10626 . Anyone who can help commit the latest patch? Thanks in advance.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Sorry for the delay Yiqun Lin. I'm committing it shortly.

          Show
          yzhangal Yongjun Zhang added a comment - Sorry for the delay Yiqun Lin . I'm committing it shortly.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10368 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10368/)
          HDFS-10625. VolumeScanner to report why a block is found bad. (yzhang: rev 5d1609ddf275e4907bd224bf618e2aad4b262888)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10368 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10368/ ) HDFS-10625 . VolumeScanner to report why a block is found bad. (yzhang: rev 5d1609ddf275e4907bd224bf618e2aad4b262888) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Hide
          yzhangal Yongjun Zhang added a comment - - edited

          Committed to trunk, branch-2, branch-2.8 and branch-2.7.

          Thanks Rushabh S Shah and Yiqun Lin for the contribution, and Vinayakumar B for the review.

          Show
          yzhangal Yongjun Zhang added a comment - - edited Committed to trunk, branch-2, branch-2.8 and branch-2.7. Thanks Rushabh S Shah and Yiqun Lin for the contribution, and Vinayakumar B for the review.

            People

            • Assignee:
              shahrs87 Rushabh S Shah
              Reporter:
              yzhangal Yongjun Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development