Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14834

in some particular situation, datanode will always in DECOMMISSION_INPROGRESS state

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.2
    • Fix Version/s: None
    • Component/s: hdfs
    • Labels:
      None
    • Environment:

      Policy:RS-6-3-1024K

      Version:3.1.2

      Description

      The file's block index is [0,1,2,3,4,5,6,7,8]. I decommission index [3] and  increase the index 5 datanode's pendingReplicationWithoutTargets.

      After reconstruction of index 5, the block status is:

      index isDecommissionInProgress state
      0                false LIVE
      1                false LIVE
      2                false LIVE
      3                true DECOMMISSIONING
      4                false LIVE
      5                false LIVE
      6                false LIVE
      7                false LIVE
      8                false LIVE
      5                false LIVE

       

      In DatanodeAdminManager.Monitor thread, blockManager.countNodes(block) caculate the live bitset is {0, 1, 2, 4, 5, 6, 7, 8}, liveReplicas: 8, redundant:1

      It's a low redundancy block, put it into queue and wait for schedule.

      In BlockManager.RedundancyMonitor thread, live bitset is {0, 1, 2, 3, 4, 6, 7, 8} , liveReplicas:9,  redundant :0

      The block waitting for replication will remove from queue, rbecause the liveReplicas satisfies the expected redundancy

       And this situation would lead to index[3]'s datanode allways in DECOMMISSION_INPROGRESS state

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              janick janick.wu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: