Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2547

ReplicationTargetChooser has incorrect block placement comments

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 1.1.0
    • Component/s: namenode
    • Labels:
      None

      Description

      /** The class is responsible for choosing the desired number of targets
       * for placing block replicas.
       * The replica placement strategy is that if the writer is on a datanode,
       * the 1st replica is placed on the local machine, 
       * otherwise a random datanode. The 2nd replica is placed on a datanode
       * that is on a different rack. The 3rd replica is placed on a datanode
       * which is on the same rack as the **first replca**.
       */
      

      That should read "second replica". The test cases confirm that this is the behavior, as well as the docs.

      1. HDFS-2547.patch
        0.8 kB
        Harsh J
      2. HDFS-2547.patch
        1 kB
        Harsh J

        Activity

        Harsh J created issue -
        Harsh J made changes -
        Field Original Value New Value
        Attachment HDFS-2547.patch [ 12503161 ]
        Harsh J made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12503161/HDFS-2547.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.TestFileAppend
        org.apache.hadoop.hdfs.TestDFSRemove
        org.apache.hadoop.hdfs.TestFileAppend2
        org.apache.hadoop.hdfs.TestBalancerBandwidth

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1550//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1550//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12503161/HDFS-2547.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestFileAppend org.apache.hadoop.hdfs.TestDFSRemove org.apache.hadoop.hdfs.TestFileAppend2 org.apache.hadoop.hdfs.TestBalancerBandwidth +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1550//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1550//console This message is automatically generated.
        Hide
        Harsh J added a comment -

        The ReplicationTargetChooser comments are incorrect, and lead to this confusion. Resolving as invalid. The documented behavior is correct for all of the common cases.

        Show
        Harsh J added a comment - The ReplicationTargetChooser comments are incorrect, and lead to this confusion. Resolving as invalid. The documented behavior is correct for all of the common cases.
        Harsh J made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Invalid [ 6 ]
        Hide
        Aaron T. Myers added a comment -

        Hey Harsh, seems like we should fix the comments then. Want to re-title/re-open this JIRA for that purpose?

        Show
        Aaron T. Myers added a comment - Hey Harsh, seems like we should fix the comments then. Want to re-title/re-open this JIRA for that purpose?
        Hide
        Harsh J added a comment -

        Aaron,

        BlockPlacementPolicyDefault has proper comments in it, on trunk. It was 0.20's ReplicationTargetChooser that threw me off there. Reopening to fix against 0.20.

        Show
        Harsh J added a comment - Aaron, BlockPlacementPolicyDefault has proper comments in it, on trunk. It was 0.20's ReplicationTargetChooser that threw me off there. Reopening to fix against 0.20.
        Harsh J made changes -
        Resolution Invalid [ 6 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Hide
        Harsh J added a comment -

        Updating title/topic for 0.20

        Show
        Harsh J added a comment - Updating title/topic for 0.20
        Harsh J made changes -
        Summary Design doc is wrong about default block placement policy. ReplicationTargetChooser has incorrect block placement comments
        Fix Version/s 0.20.206.0 [ 12317959 ]
        Fix Version/s 0.24.0 [ 12317653 ]
        Description bq. For the common case, when the replication factor is three, HDFS's placement policy is to put one replica on one node in the local rack, another on a node in a different (remote) rack, and the last on a different node in the same *remote* rack.

        Should actually be: "and the last on a different node in the same *local* rack."
        {code}
        /** The class is responsible for choosing the desired number of targets
         * for placing block replicas.
         * The replica placement strategy is that if the writer is on a datanode,
         * the 1st replica is placed on the local machine,
         * otherwise a random datanode. The 2nd replica is placed on a datanode
         * that is on a different rack. The 3rd replica is placed on a datanode
         * which is on the same rack as the **first replca**.
         */
        {code}

        That should read "second replica". The test cases confirm that this is the behavior, as well as the docs.
        Hide
        Harsh J added a comment -

        New patch that changes comments and fixes a typo.

        Show
        Harsh J added a comment - New patch that changes comments and fixes a typo.
        Harsh J made changes -
        Attachment HDFS-2547.patch [ 12503369 ]
        Harsh J made changes -
        Status Reopened [ 4 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12503369/HDFS-2547.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1553//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12503369/HDFS-2547.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1553//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        +1, patch looks good to me.

        Show
        Aaron T. Myers added a comment - +1, patch looks good to me.
        Hide
        Harsh J added a comment -

        Thanks Aaron for your persistence on this one. Committed to branch-1.

        Show
        Harsh J added a comment - Thanks Aaron for your persistence on this one. Committed to branch-1.
        Harsh J made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop-1.1.0.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop-1.1.0.
        Matt Foley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        2h 1 Harsh J 10/Nov/11 04:37
        Resolved Resolved Reopened Reopened
        1d 3h 13m 1 Harsh J 11/Nov/11 12:09
        Reopened Reopened Patch Available Patch Available
        53m 55s 1 Harsh J 11/Nov/11 13:03
        Patch Available Patch Available Resolved Resolved
        46d 2h 21m 2 Harsh J 27/Dec/11 11:06
        Resolved Resolved Closed Closed
        295d 7h 21m 1 Matt Foley 17/Oct/12 18:27

          People

          • Assignee:
            Harsh J
            Reporter:
            Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development