Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5579

Under construction files make DataNode decommission take very long hours

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We noticed that some times decommission DataNodes takes very long time, even exceeds 100 hours.
      After check the code, I found that in BlockManager:computeReplicationWorkForBlocks(List<List<Block>> blocksToReplicate) it won't replicate blocks which belongs to under construction files, however in BlockManager:isReplicationInProgress(DatanodeDescriptor srcNode), if there is block need replicate no matter whether it belongs to under construction or not, the decommission progress will continue running.
      That's the reason some time the decommission takes very long time.

        Attachments

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              zhaoyunjiong yunjiong zhao Assign to me
              Reporter:
              zhaoyunjiong yunjiong zhao

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment