Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2576

Namenode should have a favored nodes hint to enable clients to have control over block placement.

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.0-beta
    • Component/s: hdfs-client, namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Sometimes Clients like HBase are required to dynamically compute the datanodes it wishes to place the blocks for a file for higher level of locality. For this purpose there is a need of a way to give the Namenode a hint in terms of a favoredNodes parameter about the locations where the client wants to put each block. The proposed solution is a favored nodes parameter in the addBlock() method and in the create() file method to enable the clients to give the hints to the NameNode about the locations of each replica of the block. Note that this would be just a hint and finally the NameNode would look at disk usage, datanode load etc. and decide whether it can respect the hints or not.

      1. hdfs-2576-1.txt
        19 kB
        Devaraj Das
      2. hdfs-2576-trunk-1.patch
        25 kB
        Devaraj Das
      3. hdfs-2576-trunk-2.patch
        28 kB
        Devaraj Das
      4. hdfs-2576-trunk-7.1.patch
        32 kB
        Devaraj Das
      5. hdfs-2576-trunk-7.patch
        32 kB
        Devaraj Das
      6. hdfs-2576-trunk-8.1.patch
        43 kB
        Devaraj Das
      7. hdfs-2576-trunk-8.2.patch
        44 kB
        Devaraj Das
      8. hdfs-2576-trunk-8.3.patch
        44 kB
        Devaraj Das
      9. hdfs-2576-trunk-8.patch
        43 kB
        Devaraj Das

        Issue Links

          Activity

          Pritam Damania created issue -
          Andrew Purtell made changes -
          Field Original Value New Value
          Link This issue relates to HBASE-4755 [ HBASE-4755 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-1.txt [ 12568509 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-1.txt [ 12568509 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-1.txt [ 12568510 ]
          Nicolas Liochon made changes -
          Link This issue is required by HBASE-5843 [ HBASE-5843 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-trunk-1.patch [ 12572053 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-trunk-2.patch [ 12572859 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-trunk-7.patch [ 12577936 ]
          Devaraj Das made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee Devaraj Das [ devaraj ]
          Devaraj Das made changes -
          Fix Version/s 2.0.5-beta [ 12324031 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-trunk-7.1.patch [ 12577959 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-trunk-8.patch [ 12580179 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-trunk-8.1.patch [ 12580212 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-trunk-8.2.patch [ 12580648 ]
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Devaraj Das made changes -
          Attachment hdfs-2576-trunk-8.3.patch [ 12580672 ]
          Tsz Wo Nicholas Sze made changes -
          Component/s hdfs-client [ 12312928 ]
          Component/s namenode [ 12312926 ]
          Devaraj Das made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Devaraj Das made changes -
          Link This issue is related to HDFS-4779 [ HDFS-4779 ]
          Devaraj Das made changes -
          Link This issue relates to HDFS-4778 [ HDFS-4778 ]
          Colin Patrick McCabe made changes -
          Link This issue is related to HDFS-4778 [ HDFS-4778 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue is related to HDFS-6680 [ HDFS-6680 ]

            People

            • Assignee:
              Devaraj Das
              Reporter:
              Pritam Damania
            • Votes:
              1 Vote for this issue
              Watchers:
              40 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development