Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11419 BlockPlacementPolicyDefault is choosing datanode in an inefficient way
  3. HDFS-11482

Add storage type demand to into DFSNetworkTopology#chooseRandom

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha4
    • namenode
    • None
    • Reviewed

    Description

      HDFS-11450 adds storage type info into network topology, with on this info we may change chooseRandom to take storage type requirement as parameter, only checking subtrees required storage type available. This way we avoid blindly picking up nodes that are not applicable.

      Attachments

        1. HDFS-11482.001.patch
          19 kB
          Chen Liang
        2. HDFS-11482.002.patch
          18 kB
          Chen Liang
        3. HDFS-11482.003.patch
          18 kB
          Chen Liang
        4. HDFS-11482.004.patch
          19 kB
          Chen Liang
        5. HDFS-11482.005.patch
          21 kB
          Chen Liang
        6. HDFS-11482.006.patch
          23 kB
          Chen Liang
        7. HDFS-11482.007.patch
          29 kB
          Chen Liang
        8. HDFS-11482.008.patch
          28 kB
          Chen Liang
        9. HDFS-11482.009.patch
          28 kB
          Chen Liang
        10. HDFS-11482.010.patch
          28 kB
          Chen Liang

        Issue Links

        depends upon

        Sub-task - The sub-task of the issue HDFS-11450 HDFS specific network topology classes with storage type info included

        • Major - Major loss of function.
        • Resolved
        is depended upon by

        Sub-task - The sub-task of the issue HDFS-11514 DFSTopologyNodeImpl#chooseRandom optimizations

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vagarychen Chen Liang
            vagarychen Chen Liang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment