Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7197 Enhancements to Mover
  3. HDFS-8540

Mover should exit with NO_MOVE_BLOCK if no block can be moved

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • balancer & mover
    • None
    • Reviewed

    Description

      When there are files not satisfying their storage policy and no move is possible, Mover exits with SUCCESS. It should exit with NO_MOVE_BLOCK.

      The bug seems in the following code. When StorageTypeDiff is not empty and scheduleMoves4Block return false, it does not update hasRemaining. Also, there is no indication of "No block can be moved" for the entire iteration.

      //Mover.processFile(..)
              if (!diff.removeOverlap(true)) {
                if (scheduleMoves4Block(diff, lb, ecSchema)) {
                  hasRemaining |= (diff.existing.size() > 1 &&
                      diff.expected.size() > 1);
                }
              }
      

      Attachments

        1. HDFS-8540_1.patch
          13 kB
          Surendra Singh Lilhore
        2. HDFS-8540_2.patch
          13 kB
          Surendra Singh Lilhore
        3. HDFS-8540_3.patch
          13 kB
          Surendra Singh Lilhore
        4. HDFS-8540.patch
          10 kB
          Surendra Singh Lilhore

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            surendralilhore Surendra Singh Lilhore
            szetszwo Tsz-wo Sze
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment