Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21014

Improve Stochastic Balancer to write HDFS favoured node hints for region primary blocks to avoid destroying data locality if needing to use HDFS Balancer

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.1.2
    • None
    • Balancer

    Description

      Improve Stochastic Balancer to include the HDFS region location hints to avoid HDFS Balancer destroying data locality.

      Right now according to a mix of docs, jiras and mailing list info it appears that one must change

      hbase.master.loadbalancer.class

      to the org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer as it looks like this functionality is only within FavoredNodeBalancer and not the standard Stochastic Balancer.

      http://hbase.apache.org/book.html#_hbase_and_hdfs

      This is not ideal because we'd still like to use all the heuristics and work that has gone in the Stochastic Balancer which I believe right now is the best and most mature HBase balancer.

      See also the linked Jiras and this discussion:

      http://apache-hbase.679495.n3.nabble.com/HDFS-Balancer-td4086607.html

      Attachments

        Issue Links

          duplicates

          Umbrella - An overarching type made of sub-tasks HBASE-15531 Favored Nodes Enhancements

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue HBASE-7932 Do the necessary plumbing for the region locations in META table and send the info to the RegionServers

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Sub-task - The sub-task of the issue HBASE-7942 Make use of the plumbing in HBASE-7932 to provide location hints when region files are created

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-17760 HDFS Balancer doc is misleading

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6133 Add a feature for replica pinning so that a pinned replica will not be moved by Balancer/Mover.

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              harisekhon Hari Sekhon
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: