Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8586

Dead Datanode is allocated for write when client is from deadnode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • None
    • None
    • Reviewed

    Description

      DataNode marked as Dead
      2015-06-11 19:39:00,862 | INFO | org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor@28ec166e | BLOCK* removeDeadDatanode: lost heartbeat from XX.XX.39.33:25009 | org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.removeDeadDatanode(DatanodeManager.java:584)
      2015-06-11 19:39:00,863 | INFO | org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor@28ec166e | Removing a node: /default/rack3/XX.XX.39.33:25009 | org.apache.hadoop.net.NetworkTopology.remove(NetworkTopology.java:488)

      Deadnode got Allocated
      2015-06-11 19:39:45,148 | WARN | IPC Server handler 26 on 25000 | The cluster does not contain node: /default/rack3/XX.XX.39.33:25009 | org.apache.hadoop.net.NetworkTopology.getDistance(NetworkTopology.java:616)
      2015-06-11 19:39:45,149 | WARN | IPC Server handler 26 on 25000 | The cluster does not contain node: /default/rack3/XX.XX.39.33:25009 | org.apache.hadoop.net.NetworkTopology.getDistance(NetworkTopology.java:616)
      2015-06-11 19:39:45,149 | WARN | IPC Server handler 26 on 25000 | The cluster does not contain node: /default/rack3/XX.XX.39.33:25009 | org.apache.hadoop.net.NetworkTopology.getDistance(NetworkTopology.java:616)
      2015-06-11 19:39:45,149 | WARN | IPC Server handler 26 on 25000 | The cluster does not contain node: /default/rack3/XX.XX.39.33:25009 | org.apache.hadoop.net.NetworkTopology.getDistance(NetworkTopology.java:616)
      2015-06-11 19:39:45,149 | INFO | IPC Server handler 26 on 25000 | BLOCK* allocate blk_1073754030_13252 {UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-e8d29773-dfc2-4224-b1d6-9b0588bca55e:NORMAL:XX.XX.39.33:25009|RBW], ReplicaUC[[DISK]DS-f7d2ab3c-88f7-470c-9097-84387c0bec83:NORMAL:XX.XX.38.32:25009|RBW], ReplicaUC[[DISK]DS-8c2a464a-ac81-4651-890a-dbfd07ddd95f:NORMAL: XX.XX.38.33:25009|RBW]] } for /t1.COPYING | org.apache.hadoop.hdfs.server.namenode.FSNamesystem.saveAllocatedBlock(FSNamesystem.java:3657)
      2015-06-11 19:39:45,191 | INFO | IPC Server handler 35 on 25000 | BLOCK* allocate blk_1073754031_13253{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ed8ad579-50c0-4e3e-8780-9776531763b6:NORMAL:XX.XX.39.31:25009|RBW], ReplicaUC[[DISK]DS-19ddd6da-4a3e-481a-8445-dde5c90aaff3:NORMAL:XX.XX.37.32:25009|RBW], ReplicaUC[[DISK]DS-4ce4ce39-4973-42ce-8c7d-cb41f899db85: NORMAL:XX.XX.37.33:25009 |RBW]]} for /t1.COPYING | org.apache.hadoop.hdfs.server.namenode.FSNamesystem.saveAllocatedBlock(FSNamesystem.java:3657)

      Attachments

        1. HDFS-8586.patch
          5 kB
          Brahma Reddy Battula

        Activity

          People

            brahmareddy Brahma Reddy Battula
            brahmareddy Brahma Reddy Battula
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: