Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-12731

Heap occupancy based client pushback

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.0.0, 0.98.10, 1.1.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      This feature incorporates reported regionserver heap occupancy in the (optional) client pushback calculations. If client pushback is enabled, the exponential backoff policy will take heap occupancy into account, should it exceed "hbase.heap.occupancy.low_water_mark" percentage of the heap (default 0.95). Once above the low water mark, heap occupancy is an additional factor scaling from 0.1 up to 1.0 at "hbase.heap.occupancy.high_water_mark" (default 0.98). At or above the high water mark the client will use the maximum configured backoff.
      Show
      This feature incorporates reported regionserver heap occupancy in the (optional) client pushback calculations. If client pushback is enabled, the exponential backoff policy will take heap occupancy into account, should it exceed "hbase.heap.occupancy.low_water_mark" percentage of the heap (default 0.95). Once above the low water mark, heap occupancy is an additional factor scaling from 0.1 up to 1.0 at "hbase.heap.occupancy.high_water_mark" (default 0.98). At or above the high water mark the client will use the maximum configured backoff.

      Description

      If the heap occupancy of a RegionServer is beyond a configurable high water mark (suggestions: 95%, 98%) then we should reject all user RPCs and only allow administrative RPCs until occupancy has dropped below a configurable low water mark (suggestions: 92%).

      Implement building on the HBASE-5162 changes.

      It might be expensive to check heap occupancy, in which case we can sample it periodically with a chore and use the last known value in pushback calculations.

        Attachments

        1. HBASE-12731.patch
          29 kB
          Andrew Kyle Purtell
        2. HBASE-12731.patch
          35 kB
          Andrew Kyle Purtell
        3. HBASE-12731.patch
          36 kB
          Andrew Kyle Purtell
        4. HBASE-12731-0.98.patch
          36 kB
          Andrew Kyle Purtell
        5. HBASE-12731-0.98.patch
          36 kB
          Andrew Kyle Purtell
        6. HBASE-12731.patch
          36 kB
          Andrew Kyle Purtell

          Issue Links

            Activity

              People

              • Assignee:
                apurtell Andrew Kyle Purtell
                Reporter:
                apurtell Andrew Kyle Purtell
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: