Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15

Rack replication policy can be violated for over replicated blocks

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.20.3
    • Fix Version/s: 0.20.3, 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      HDFS replicas placement strategy guarantees that the replicas of a block exist on at least two racks when its replication factor is greater than one. But fsck still reports that the replicas of some blocks end up on one rack.

      The cause of the problem is that decommission and corruption handling only check the block's replication factor but not the rack requirement. When an over-replicated block loses a replica due to decomission, corruption, or heartbeat lost, namenode does not take any action to guarantee that remaining replicas are on different racks.

        Attachments

        1. hdfs-15-b20-1.patch
          34 kB
          Eli Collins
        2. HDFS-15.6.patch
          19 kB
          Jitendra Nath Pandey
        3. HDFS-15.5.patch
          19 kB
          Jitendra Nath Pandey
        4. HDFS-15.4.patch
          18 kB
          Jitendra Nath Pandey
        5. HDFS-15.patch.3
          18 kB
          Jitendra Nath Pandey
        6. HDFS-15.patch.2
          17 kB
          Jitendra Nath Pandey
        7. HDFS-15.patch
          25 kB
          Jitendra Nath Pandey

          Issue Links

            Activity

              People

              • Assignee:
                jnp Jitendra Nath Pandey
                Reporter:
                hairong Hairong Kuang
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: