Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15760

Validate the target indices in ErasureCoding worker in reconstruction process

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.4.0
    • Fix Version/s: None
    • Component/s: ec
    • Labels:
      None

      Description

      As we have seen issues like 

      1. HDFS-15186
      2. HDFS-14768

      It is a good idea to validate the indices at the ECWorker side and skip the unintended in indices from target list.

      Both of the issues triggered because, NN accidentally scheduled for reconstruction in decom process due to busy node. We have fixed to make sure NN considers busy nodes as live replicas. However, it may be good idea to safe gaud the condition at ECWorker also in case if any other condition triggers and that leads ECWroker to calculate the indices similar the above issues, then EC function returns wrong o/p.  I think it's ok to recover only the missing indices from the given src indices.

       

        Attachments

          Activity

            People

            • Assignee:
              umamaheswararao Uma Maheswara Rao G
              Reporter:
              umamaheswararao Uma Maheswara Rao G

              Dates

              • Created:
                Updated:

                Issue deployment