Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1765

Block Replication should respect under-replication block priority

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 2.0.0-alpha, 0.23.7
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently under-replicated blocks are assigned different priorities depending on how many replicas a block has. However the replication monitor works on blocks in a round-robin fashion. So the newly added high priority blocks won't get replicated until all low-priority blocks are done. One example is that on decommissioning datanode WebUI we often observe that "blocks with only decommissioning replicas" do not get scheduled to replicate before other blocks, so risking data availability if the node is shutdown for repair before decommission completes.

      1. underReplicatedQueue.pdf
        25 kB
        Haryadi Gunawi
      2. HDFS-1765.pdf
        560 kB
        Uma Maheswara Rao G
      3. HDFS-1765.patch
        17 kB
        Uma Maheswara Rao G
      4. HDFS-1765.patch
        19 kB
        Uma Maheswara Rao G
      5. HDFS-1765.patch
        22 kB
        Uma Maheswara Rao G
      6. HDFS-1765.patch
        22 kB
        Uma Maheswara Rao G

        Issue Links

          Activity

            People

            • Assignee:
              Uma Maheswara Rao G
              Reporter:
              Hairong Kuang
            • Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h

                  Development