Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-385

Design a pluggable interface to place replicas of blocks in HDFS

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0, 1.2.0, 1-win
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      New experimental API BlockPlacementPolicy allows investigating alternate rules for locating block replicas.

      Description

      The current HDFS code typically places one replica on local rack, the second replica on remote random rack and the third replica on a random node of that remote rack. This algorithm is baked in the NameNode's code. It would be nice to make the block placement algorithm a pluggable interface. This will allow experimentation of different placement algorithms based on workloads, availability guarantees and failure models.

      1. blockplacementpolicy3-branch-1-win.patch
        84 kB
        Sumadhur Reddy Bolli
      2. blockplacementpolicy3-branch-1.patch
        85 kB
        Sumadhur Reddy Bolli
      3. blockplacementpolicy2-branch-1-win.patch
        84 kB
        Sumadhur Reddy Bolli
      4. blockplacementpolicy2-branch-1.patch
        85 kB
        Sumadhur Reddy Bolli
      5. rat094.txt
        0.4 kB
        stack
      6. blockplacementpolicy-branch-1-win.patch
        27 kB
        Sumadhur Reddy Bolli
      7. blockplacementpolicy-branch-1.patch
        28 kB
        Sumadhur Reddy Bolli
      8. BlockPlacementPluggable7.txt
        77 kB
        dhruba borthakur
      9. BlockPlacementPluggable6.txt
        77 kB
        dhruba borthakur
      10. BlockPlacementPluggable5.txt
        79 kB
        dhruba borthakur
      11. BlockPlacementPluggable4.txt
        79 kB
        dhruba borthakur
      12. BlockPlacementPluggable4.txt
        76 kB
        dhruba borthakur
      13. BlockPlacementPluggable3.txt
        70 kB
        dhruba borthakur
      14. BlockPlacementPluggable2.txt
        61 kB
        dhruba borthakur
      15. BlockPlacementPluggable.txt
        16 kB
        dhruba borthakur

        Issue Links

          Activity

            People

            • Assignee:
              dhruba borthakur
              Reporter:
              dhruba borthakur
            • Votes:
              4 Vote for this issue
              Watchers:
              35 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development