Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14849

Erasure Coding: the internal block is replicated many times when datanode is decommissioning

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      colored textWhen the datanode keeping in DECOMMISSION_INPROGRESS status, the EC internal block in that datanode will be replicated many times.

      // added 2019/09/19
      I reproduced this scenario in a 163 nodes cluster with decommission 100 nodes simultaneously.

        Attachments

        1. fsck-file.png
          2.89 MB
          HuangTao
        2. HDFS-14849.001.patch
          5 kB
          HuangTao
        3. HDFS-14849.002.patch
          5 kB
          HuangTao
        4. HDFS-14849.branch-3.1.patch
          5 kB
          Wei-Chiu Chuang
        5. liveBlockIndices.png
          450 kB
          HuangTao
        6. scheduleReconstruction.png
          124 kB
          HuangTao

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment