HBase
  1. HBase
  2. HBASE-3455

Heap fragmentation in region server

    Details

    • Type: Brainstorming Brainstorming
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.90.1
    • Fix Version/s: 0.90.1
    • Component/s: Performance, regionserver
    • Labels:
      None
    • Release Note:
      Adds an allocation buffer local to each memstore which is used to combat heap fragmentation. Experimental in 0.90.1 - enable by toggling hbase.hregion.memstore.mslab.enabled. Enabled by default in 0.91

      Description

      Stop-the-world GC pauses have long been a problem in HBase. "Concurrent mode failures" can usually be tuned around by setting the initiating occupancy fraction low, but eventually the heap becomes fragmented and a promotion failure occurs.

      This JIRA is to do research/experiments about the heap fragmentation issue and possible solutions.

      1. with-kvallocs.png
        79 kB
        Todd Lipcon
      2. parse-fls-statistics.py
        0.9 kB
        Todd Lipcon
      3. mslab-6.txt
        27 kB
        Todd Lipcon
      4. mslab-5.txt
        26 kB
        Todd Lipcon
      5. mslab-4.txt
        23 kB
        Todd Lipcon
      6. mslab-3.txt
        23 kB
        Todd Lipcon
      7. mslab-2.txt
        22 kB
        Todd Lipcon
      8. mslab-1.txt
        17 kB
        Todd Lipcon
      9. icv-frag.png
        109 kB
        Todd Lipcon
      10. HBasefragmentation.pdf
        194 kB
        Todd Lipcon
      11. collapse-arrays.patch
        6 kB
        Todd Lipcon

        Activity

          People

          • Assignee:
            Todd Lipcon
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            18 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development