Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14578

AvailableSpaceBlockPlacementPolicy always prefers local node

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Fix Version/s: 3.3.0, 3.1.4, 3.2.2
    • Component/s: block placement
    • Labels:
      None

      Description

      It looks like AvailableSpaceBlockPlacementPolicy prefers local disk just like in the BlockPlacementPolicyDefault

       

      As Yongjun mentioned in HDFS-8131

      Class AvailableSpaceBlockPlacementPolicy extends BlockPlacementPolicyDefault. But it doesn't change the behavior of choosing the first node in BlockPlacementPolicyDefault, so even with this new feature, the local DN is always chosen as the first DN (of course when it is not excluded), and the new feature only changes the selection of the rest of the two DNs.

      I'm file this Jira as I groom Cloudera's internal Jira and found this unreported issue. We do have a customer hitting this problem. I don't have a fix, but thought it would be beneficial to report it to Apache Jira.

        Attachments

        1. HDFS-14578-07.patch
          15 kB
          Ayush Saxena
        2. HDFS-14578-06.patch
          14 kB
          Ayush Saxena
        3. HDFS-14578-05.patch
          14 kB
          Ayush Saxena
        4. HDFS-14578-04.patch
          14 kB
          Ayush Saxena
        5. HDFS-14578-03.patch
          14 kB
          Ayush Saxena
        6. HDFS-14578-02.patch
          14 kB
          Ayush Saxena
        7. HDFS-14758-01.patch
          12 kB
          Ayush Saxena
        8. HDFS-14578-WIP-01.patch
          7 kB
          Ayush Saxena

          Activity

            People

            • Assignee:
              ayushtkn Ayush Saxena
              Reporter:
              weichiu Wei-Chiu Chuang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: