Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-3455

Heap fragmentation in region server

    XMLWordPrintableJSON

Details

    • Brainstorming
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.90.1
    • 0.90.1
    • Performance, regionserver
    • None
    • Adds an allocation buffer local to each memstore which is used to combat heap fragmentation. Experimental in 0.90.1 - enable by toggling hbase.hregion.memstore.mslab.enabled. Enabled by default in 0.91

    Description

      Stop-the-world GC pauses have long been a problem in HBase. "Concurrent mode failures" can usually be tuned around by setting the initiating occupancy fraction low, but eventually the heap becomes fragmented and a promotion failure occurs.

      This JIRA is to do research/experiments about the heap fragmentation issue and possible solutions.

      Attachments

        1. mslab-6.txt
          27 kB
          Todd Lipcon
        2. mslab-5.txt
          26 kB
          Todd Lipcon
        3. mslab-4.txt
          23 kB
          Todd Lipcon
        4. mslab-3.txt
          23 kB
          Todd Lipcon
        5. mslab-2.txt
          22 kB
          Todd Lipcon
        6. icv-frag.png
          109 kB
          Todd Lipcon
        7. mslab-1.txt
          17 kB
          Todd Lipcon
        8. with-kvallocs.png
          79 kB
          Todd Lipcon
        9. collapse-arrays.patch
          6 kB
          Todd Lipcon
        10. parse-fls-statistics.py
          0.9 kB
          Todd Lipcon
        11. HBasefragmentation.pdf
          194 kB
          Todd Lipcon

        Activity

          People

            tlipcon Todd Lipcon
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            20 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: