Hadoop Common
  1. Hadoop Common
  2. HADOOP-7013

Add boolean field isCorrupt to BlockLocation

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      This patch has changed the serialization format of BlockLocation.

      Description

      This is needed to allow DFSClient.getBlockLocations to notify the calling application when returning a BlockLocation that corresponds to a corrupt block. Currently, this happens when there are no uncorrupted replicas of a requested block.

      1. HADOOP-7013.patch
        3 kB
        Patrick Kling

        Issue Links

          Activity

          Hide
          Patrick Kling added a comment -
          Show
          Patrick Kling added a comment - Review board: https://reviews.apache.org/r/26/
          Hide
          Hairong Kuang added a comment -

          The patch looks good except for an incompatible change.

          BlockLocation is not used in any wire-protocol, but for some reason it is Writable. So if by any chance somebody serialize this object in a file etc, then this patch makes it incompatible.

          I would suggest that this patch does not change the serialization format. Since Hadoop does not use it over the wire, it won't do any harm.

          Show
          Hairong Kuang added a comment - The patch looks good except for an incompatible change. BlockLocation is not used in any wire-protocol, but for some reason it is Writable. So if by any chance somebody serialize this object in a file etc, then this patch makes it incompatible. I would suggest that this patch does not change the serialization format. Since Hadoop does not use it over the wire, it won't do any harm.
          Hide
          Hairong Kuang added a comment -

          On a second thought, Hadoop does not have a story for Writable backward compatibility yet. So this patch does not need to take care of this either. Instead we could simply mark this jira as incompatible.

          So a +1 from me.

          Show
          Hairong Kuang added a comment - On a second thought, Hadoop does not have a story for Writable backward compatibility yet. So this patch does not need to take care of this either. Instead we could simply mark this jira as incompatible. So a +1 from me.
          Hide
          Patrick Kling added a comment -

          ant test-path results:

               [exec] -1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
               [exec]                         Please justify why no new tests are needed for this patch.
               [exec]                         Also please list what manual steps were performed to verify this patch.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     -1 findbugs.  The patch appears to introduce 2 new Findbugs warnings.
               [exec] 
               [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
               [exec] 
               [exec]     +1 system test framework.  The patch passed system test framework compile.
          

          Tests are included with HDFS-1483. The findbugs warning are not caused by this patch (see MAPREDUCE-2172 for details).

          Show
          Patrick Kling added a comment - ant test-path results: [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 2 new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. Tests are included with HDFS-1483 . The findbugs warning are not caused by this patch (see MAPREDUCE-2172 for details).
          Hide
          Eli Collins added a comment -

          +1

          Patch looks good to me.

          Show
          Eli Collins added a comment - +1 Patch looks good to me.
          Hide
          Hairong Kuang added a comment -

          I've just committed this. Thanks, Patrick!

          Show
          Hairong Kuang added a comment - I've just committed this. Thanks, Patrick!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #428 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/428/)
          HADOOP-7013. Add boolean field isCorrupt to BlockLocation. Contributed by Patrick Kling.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #428 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/428/ ) HADOOP-7013 . Add boolean field isCorrupt to BlockLocation. Contributed by Patrick Kling.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #515 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/515/)
          HADOOP-7013. Add boolean field isCorrupt to BlockLocation. Contributed by Patrick Kling.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #515 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/515/ ) HADOOP-7013 . Add boolean field isCorrupt to BlockLocation. Contributed by Patrick Kling.

            People

            • Assignee:
              Patrick Kling
              Reporter:
              Patrick Kling
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development