Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-4755 HBase based block placement in DFS
  3. HBASE-7932

Do the necessary plumbing for the region locations in META table and send the info to the RegionServers

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.1
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      Introduces "favored nodes" for region files. Has an implementation of LoadBalancer that assigns favored nodes for each region. There is a Primary RegionServer that hosts the region, and then there is Secondary and Tertiary RegionServers. Currently, the favored nodes information is used in creating HDFS files - the Primary RegionServer passes the primary, secondary, tertiary node addresses as hints to the DistributedFileSystem API for creating files on the filesystem. These nodes are treated as hints by the HDFS to place the blocks of the file (this will be complete when HBASE-7942 is addressed). This alleviates the problem to do with reading from remote nodes (since we can make the Secondary RegionServer as the new Primary RegionServer) after a region is recovered. This should help provide consistent read latencies for the regions even when their primary region servers die. The implementation of the balancer is not complete as of this issue.
      Show
      Introduces "favored nodes" for region files. Has an implementation of LoadBalancer that assigns favored nodes for each region. There is a Primary RegionServer that hosts the region, and then there is Secondary and Tertiary RegionServers. Currently, the favored nodes information is used in creating HDFS files - the Primary RegionServer passes the primary, secondary, tertiary node addresses as hints to the DistributedFileSystem API for creating files on the filesystem. These nodes are treated as hints by the HDFS to place the blocks of the file (this will be complete when HBASE-7942 is addressed). This alleviates the problem to do with reading from remote nodes (since we can make the Secondary RegionServer as the new Primary RegionServer) after a region is recovered. This should help provide consistent read latencies for the regions even when their primary region servers die. The implementation of the balancer is not complete as of this issue.
    • Tags:
      0.96notable

      Description

      To be specific, this jira is to do the following during createTable:
      1. Favored node calculation
      2. Persisting of calculation in .META.
      3. Letting RegionServers know of the favored nodes for new regions
      4. Addition of a balancer that will honor favored nodes but it is currently a noop.

        Attachments

        1. 7932-1.patch
          124 kB
          Devaraj Das
        2. 7932-3.patch
          116 kB
          Devaraj Das
        3. 7932-8.1.patch
          130 kB
          Devaraj Das
        4. 7932-8.patch
          132 kB
          Devaraj Das
        5. 7932-simple-2.patch
          38 kB
          Devaraj Das
        6. 7932-simple-with-meta-pb.1.txt
          138 kB
          Devaraj Das
        7. 7932-simple-with-meta-pb.2.txt
          138 kB
          Devaraj Das
        8. 7932-simple-with-meta-pb.6.txt
          155 kB
          Devaraj Das
        9. 7932-wip.patch
          123 kB
          Devaraj Das
        10. 7932-wip-2.patch
          122 kB
          Devaraj Das
        11. 7932-wip-3.patch
          124 kB
          Devaraj Das
        12. 7932-wip-4.patch
          122 kB
          Devaraj Das

          Issue Links

            Activity

              People

              • Assignee:
                devaraj Devaraj Das
                Reporter:
                devaraj Devaraj Das
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: