Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14578

AvailableSpaceBlockPlacementPolicy always prefers local node

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0, 2.7.4, 3.0.0-alpha1
    • 3.3.0, 3.1.4, 3.2.2
    • block placement
    • None

    Description

      It looks like AvailableSpaceBlockPlacementPolicy prefers local disk just like in the BlockPlacementPolicyDefault

       

      As Yongjun mentioned in HDFS-8131

      Class AvailableSpaceBlockPlacementPolicy extends BlockPlacementPolicyDefault. But it doesn't change the behavior of choosing the first node in BlockPlacementPolicyDefault, so even with this new feature, the local DN is always chosen as the first DN (of course when it is not excluded), and the new feature only changes the selection of the rest of the two DNs.

      I'm file this Jira as I groom Cloudera's internal Jira and found this unreported issue. We do have a customer hitting this problem. I don't have a fix, but thought it would be beneficial to report it to Apache Jira.

      Attachments

        1. HDFS-14578-WIP-01.patch
          7 kB
          Ayush Saxena
        2. HDFS-14758-01.patch
          12 kB
          Ayush Saxena
        3. HDFS-14578-02.patch
          14 kB
          Ayush Saxena
        4. HDFS-14578-03.patch
          14 kB
          Ayush Saxena
        5. HDFS-14578-04.patch
          14 kB
          Ayush Saxena
        6. HDFS-14578-05.patch
          14 kB
          Ayush Saxena
        7. HDFS-14578-06.patch
          14 kB
          Ayush Saxena
        8. HDFS-14578-07.patch
          15 kB
          Ayush Saxena

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ayushtkn Ayush Saxena
            weichiu Wei-Chiu Chuang
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment