Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11419 BlockPlacementPolicyDefault is choosing datanode in an inefficient way
  3. HDFS-11530

Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha2
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: namenode
    • Labels:
      None

      Description

      The work for chooseRandomWithStorageType has been merged in HDFS-11482. But this method is contained in new topology DFSNetworkTopology which is specified for HDFS. We should update this and let BlockPlacementPolicyDefault use the new way since the original way is inefficient.

        Attachments

        1. HDFS-11530.001.patch
          3 kB
          Yiqun Lin
        2. HDFS-11530.002.patch
          5 kB
          Yiqun Lin
        3. HDFS-11530.003.patch
          12 kB
          Chen Liang
        4. HDFS-11530.004.patch
          13 kB
          Chen Liang
        5. HDFS-11530.005.patch
          12 kB
          Yiqun Lin
        6. HDFS-11530.006.patch
          12 kB
          Yiqun Lin
        7. HDFS-11530.007.patch
          13 kB
          Yiqun Lin
        8. HDFS-11530.008.patch
          18 kB
          Chen Liang
        9. HDFS-11530.009.patch
          18 kB
          Chen Liang
        10. HDFS-11530.010.patch
          31 kB
          Yiqun Lin
        11. HDFS-11530.011.patch
          34 kB
          Yiqun Lin
        12. HDFS-11530.012.patch
          33 kB
          Yiqun Lin
        13. HDFS-11530.013.patch
          34 kB
          Yiqun Lin
        14. HDFS-11530.014.patch
          23 kB
          Yiqun Lin
        15. HDFS-11530.015.patch
          23 kB
          Yiqun Lin

          Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-12272 chooseRemoteRack() semantics broken in trunk

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13418 NetworkTopology should be configurable when enable DFSNetworkTopology

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                linyiqun Yiqun Lin
                Reporter:
                linyiqun Yiqun Lin
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: