Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11419 BlockPlacementPolicyDefault is choosing datanode in an inefficient way
  3. HDFS-11530

Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha2
    • 2.9.0, 3.0.0-alpha4
    • namenode
    • None

    Description

      The work for chooseRandomWithStorageType has been merged in HDFS-11482. But this method is contained in new topology DFSNetworkTopology which is specified for HDFS. We should update this and let BlockPlacementPolicyDefault use the new way since the original way is inefficient.

      Attachments

        1. HDFS-11530.001.patch
          3 kB
          Yiqun Lin
        2. HDFS-11530.002.patch
          5 kB
          Yiqun Lin
        3. HDFS-11530.003.patch
          12 kB
          Chen Liang
        4. HDFS-11530.004.patch
          13 kB
          Chen Liang
        5. HDFS-11530.005.patch
          12 kB
          Yiqun Lin
        6. HDFS-11530.006.patch
          12 kB
          Yiqun Lin
        7. HDFS-11530.007.patch
          13 kB
          Yiqun Lin
        8. HDFS-11530.008.patch
          18 kB
          Chen Liang
        9. HDFS-11530.009.patch
          18 kB
          Chen Liang
        10. HDFS-11530.010.patch
          31 kB
          Yiqun Lin
        11. HDFS-11530.011.patch
          34 kB
          Yiqun Lin
        12. HDFS-11530.012.patch
          33 kB
          Yiqun Lin
        13. HDFS-11530.013.patch
          34 kB
          Yiqun Lin
        14. HDFS-11530.014.patch
          23 kB
          Yiqun Lin
        15. HDFS-11530.015.patch
          23 kB
          Yiqun Lin

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-12272 chooseRemoteRack() semantics broken in trunk

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13418 NetworkTopology should be configurable when enable DFSNetworkTopology

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              linyiqun Yiqun Lin
              linyiqun Yiqun Lin
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: