Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15

Rack replication policy can be violated for over replicated blocks

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.20.3
    • 0.20.3, 0.21.0
    • None
    • None

    Description

      HDFS replicas placement strategy guarantees that the replicas of a block exist on at least two racks when its replication factor is greater than one. But fsck still reports that the replicas of some blocks end up on one rack.

      The cause of the problem is that decommission and corruption handling only check the block's replication factor but not the rack requirement. When an over-replicated block loses a replica due to decomission, corruption, or heartbeat lost, namenode does not take any action to guarantee that remaining replicas are on different racks.

      Attachments

        1. HDFS-15.patch
          25 kB
          Jitendra Nath Pandey
        2. HDFS-15.patch.2
          17 kB
          Jitendra Nath Pandey
        3. HDFS-15.patch.3
          18 kB
          Jitendra Nath Pandey
        4. HDFS-15.4.patch
          18 kB
          Jitendra Nath Pandey
        5. HDFS-15.5.patch
          19 kB
          Jitendra Nath Pandey
        6. HDFS-15.6.patch
          19 kB
          Jitendra Nath Pandey
        7. hdfs-15-b20-1.patch
          34 kB
          Eli Collins

        Issue Links

          Activity

            People

              jnp Jitendra Nath Pandey
              hairong Hairong Kuang
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: