Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9083

Replication violates block placement policy.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.7.2, 2.6.3
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Recently we are noticing many cases in which all the replica of the block are residing on the same rack.
      During the block creation, the block placement policy was honored.
      But after node failure event in some specific manner, the block ends up in such state.

      On investigating more I found out that BlockManager#blockHasEnoughRacks is dependent on the config (net.topology.script.file.name)

       if (!this.shouldCheckForEnoughRacks) {
            return true;
          }
      

      We specify DNSToSwitchMapping implementation (our own custom implementation) via net.topology.node.switch.mapping.impl and no longer use net.topology.script.file.name config.

        Attachments

        1. HDFS-9083-branch-2.6.patch
          4 kB
          Rushabh S Shah
        2. HDFS-9083-branch-2.7.patch
          4 kB
          Rushabh S Shah
        3. HDFS-9083-Test fix-branch-2.7.patch
          2 kB
          Brahma Reddy Battula

          Activity

            People

            • Assignee:
              shahrs87 Rushabh S Shah
              Reporter:
              shahrs87 Rushabh S Shah
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: