Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11419 BlockPlacementPolicyDefault is choosing datanode in an inefficient way
  3. HDFS-11450

HDFS specific network topology classes with storage type info included

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This JIRA adds storage type info into network topology.

      More specifically, this JIRA adds a storage type map by extending InnerNodeImpl to describe the available storages under the current node's subtree. This map is updated when a node is added/removed from the subtree.

      With this info, when choosing a random node with storage type requirement, the search could then decide to/not to go deeper into a subtree by examining the available storage types first.

      One to-do item still, is that, we might still need to separately handle the cases where a Datanodes restarts, or a disk is hot-swapped, will file another JIRA in that case.

        Attachments

        1. HDFS-11450.001.patch
          31 kB
          Chen Liang
        2. HDFS-11450.002.patch
          34 kB
          Chen Liang
        3. HDFS-11450.003.patch
          34 kB
          Chen Liang
        4. HDFS-11450.004.patch
          34 kB
          Chen Liang

          Issue Links

            Activity

              People

              • Assignee:
                vagarychen Chen Liang
                Reporter:
                vagarychen Chen Liang
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: