Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-3455

Heap fragmentation in region server

    Details

    • Type: Brainstorming
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.90.1
    • Fix Version/s: 0.90.1
    • Component/s: Performance, regionserver
    • Labels:
      None
    • Release Note:
      Adds an allocation buffer local to each memstore which is used to combat heap fragmentation. Experimental in 0.90.1 - enable by toggling hbase.hregion.memstore.mslab.enabled. Enabled by default in 0.91

      Description

      Stop-the-world GC pauses have long been a problem in HBase. "Concurrent mode failures" can usually be tuned around by setting the initiating occupancy fraction low, but eventually the heap becomes fragmented and a promotion failure occurs.

      This JIRA is to do research/experiments about the heap fragmentation issue and possible solutions.

        Attachments

        1. mslab-6.txt
          27 kB
          Todd Lipcon
        2. mslab-5.txt
          26 kB
          Todd Lipcon
        3. mslab-4.txt
          23 kB
          Todd Lipcon
        4. mslab-3.txt
          23 kB
          Todd Lipcon
        5. mslab-2.txt
          22 kB
          Todd Lipcon
        6. icv-frag.png
          109 kB
          Todd Lipcon
        7. mslab-1.txt
          17 kB
          Todd Lipcon
        8. with-kvallocs.png
          79 kB
          Todd Lipcon
        9. collapse-arrays.patch
          6 kB
          Todd Lipcon
        10. parse-fls-statistics.py
          0.9 kB
          Todd Lipcon
        11. HBasefragmentation.pdf
          194 kB
          Todd Lipcon

          Activity

            People

            • Assignee:
              tlipcon Todd Lipcon
              Reporter:
              tlipcon Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: