Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2576

Namenode should have a favored nodes hint to enable clients to have control over block placement.

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.1.0-beta
    • hdfs-client, namenode
    • None
    • Reviewed

    Description

      Sometimes Clients like HBase are required to dynamically compute the datanodes it wishes to place the blocks for a file for higher level of locality. For this purpose there is a need of a way to give the Namenode a hint in terms of a favoredNodes parameter about the locations where the client wants to put each block. The proposed solution is a favored nodes parameter in the addBlock() method and in the create() file method to enable the clients to give the hints to the NameNode about the locations of each replica of the block. Note that this would be just a hint and finally the NameNode would look at disk usage, datanode load etc. and decide whether it can respect the hints or not.

      Attachments

        1. hdfs-2576-1.txt
          19 kB
          Devaraj Das
        2. hdfs-2576-trunk-1.patch
          25 kB
          Devaraj Das
        3. hdfs-2576-trunk-2.patch
          28 kB
          Devaraj Das
        4. hdfs-2576-trunk-7.patch
          32 kB
          Devaraj Das
        5. hdfs-2576-trunk-7.1.patch
          32 kB
          Devaraj Das
        6. hdfs-2576-trunk-8.patch
          43 kB
          Devaraj Das
        7. hdfs-2576-trunk-8.1.patch
          43 kB
          Devaraj Das
        8. hdfs-2576-trunk-8.2.patch
          44 kB
          Devaraj Das
        9. hdfs-2576-trunk-8.3.patch
          44 kB
          Devaraj Das

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ddas Devaraj Das
            blackpearl Pritam Damania
            Votes:
            1 Vote for this issue
            Watchers:
            43 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment