Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11419 BlockPlacementPolicyDefault is choosing datanode in an inefficient way
  3. HDFS-11450

HDFS specific network topology classes with storage type info included

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha4
    • namenode
    • None
    • Reviewed

    Description

      This JIRA adds storage type info into network topology.

      More specifically, this JIRA adds a storage type map by extending InnerNodeImpl to describe the available storages under the current node's subtree. This map is updated when a node is added/removed from the subtree.

      With this info, when choosing a random node with storage type requirement, the search could then decide to/not to go deeper into a subtree by examining the available storage types first.

      One to-do item still, is that, we might still need to separately handle the cases where a Datanodes restarts, or a disk is hot-swapped, will file another JIRA in that case.

      Attachments

        1. HDFS-11450.001.patch
          31 kB
          Chen Liang
        2. HDFS-11450.002.patch
          34 kB
          Chen Liang
        3. HDFS-11450.003.patch
          34 kB
          Chen Liang
        4. HDFS-11450.004.patch
          34 kB
          Chen Liang

        Issue Links

          Activity

            People

              vagarychen Chen Liang
              vagarychen Chen Liang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: