Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-15

Rack replication policy can be violated for over replicated blocks

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.20.3
    • Fix Version/s: 0.20.3, 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      HDFS replicas placement strategy guarantees that the replicas of a block exist on at least two racks when its replication factor is greater than one. But fsck still reports that the replicas of some blocks end up on one rack.

      The cause of the problem is that decommission and corruption handling only check the block's replication factor but not the rack requirement. When an over-replicated block loses a replica due to decomission, corruption, or heartbeat lost, namenode does not take any action to guarantee that remaining replicas are on different racks.

      1. HDFS-15.4.patch
        18 kB
        Jitendra Nath Pandey
      2. HDFS-15.5.patch
        19 kB
        Jitendra Nath Pandey
      3. HDFS-15.6.patch
        19 kB
        Jitendra Nath Pandey
      4. HDFS-15.patch
        25 kB
        Jitendra Nath Pandey
      5. HDFS-15.patch.2
        17 kB
        Jitendra Nath Pandey
      6. HDFS-15.patch.3
        18 kB
        Jitendra Nath Pandey
      7. hdfs-15-b20-1.patch
        34 kB
        Eli Collins

        Issue Links

          Activity

            People

            • Assignee:
              Jitendra Nath Pandey
              Reporter:
              Hairong Kuang
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development