Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17462

Use sliding window for read/write request costs in StochasticLoadBalancer

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0, 2.0.0
    • None
    • Reviewed

    Description

      In the thread, http://search-hadoop.com/m/HBase/YGbbyUZKXWALkX1, Timothy was asking whether the read/write request costs in StochasticLoadBalancer should be calculated as rates.

      This makes sense since read / write load on region server tends to fluctuate over time. Using sliding window would reflect more recent trend in read / write load.

      Some factors to consider:

      The data structure used by StochasticLoadBalancer should be concise. The
      number of regions in a cluster can be expected to approach 1 million. We
      cannot afford to store long history of read / write requests in master.

      Efficiency of cost calculation should be high - there're many cost
      functions the balancer goes through, it is expected for each cost function
      to return quickly. Otherwise we would not come up with proper region
      movement plan(s) in time.

      Attachments

        1. HBASE-17462.patch
          6 kB
          Tim Brown
        2. before_changes.png
          312 kB
          Tim Brown
        3. after_changes.png
          129 kB
          Tim Brown
        4. HBASE-17462-v2.patch
          6 kB
          Tim Brown
        5. 17462.branch-1.v2.txt
          5 kB
          Ted Yu

        Activity

          People

            timbrown Tim Brown
            yuzhihong@gmail.com Ted Yu
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: