Details

    • Type: Brainstorming Brainstorming
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.90.1
    • Fix Version/s: 0.90.1
    • Component/s: Performance, regionserver
    • Labels:
      None
    • Release Note:
      Adds an allocation buffer local to each memstore which is used to combat heap fragmentation. Experimental in 0.90.1 - enable by toggling hbase.hregion.memstore.mslab.enabled. Enabled by default in 0.91

      Description

      Stop-the-world GC pauses have long been a problem in HBase. "Concurrent mode failures" can usually be tuned around by setting the initiating occupancy fraction low, but eventually the heap becomes fragmented and a promotion failure occurs.

      This JIRA is to do research/experiments about the heap fragmentation issue and possible solutions.

      1. with-kvallocs.png
        79 kB
        Todd Lipcon
      2. parse-fls-statistics.py
        0.9 kB
        Todd Lipcon
      3. mslab-6.txt
        27 kB
        Todd Lipcon
      4. mslab-5.txt
        26 kB
        Todd Lipcon
      5. mslab-4.txt
        23 kB
        Todd Lipcon
      6. mslab-3.txt
        23 kB
        Todd Lipcon
      7. mslab-2.txt
        22 kB
        Todd Lipcon
      8. mslab-1.txt
        17 kB
        Todd Lipcon
      9. icv-frag.png
        109 kB
        Todd Lipcon
      10. HBasefragmentation.pdf
        194 kB
        Todd Lipcon
      11. collapse-arrays.patch
        6 kB
        Todd Lipcon

        Activity

        Todd Lipcon created issue -
        Todd Lipcon made changes -
        Field Original Value New Value
        Attachment HBasefragmentation.pdf [ 12468868 ]
        stack made changes -
        Priority Major [ 3 ] Critical [ 2 ]
        Todd Lipcon made changes -
        Attachment parse-fls-statistics.py [ 12468869 ]
        Todd Lipcon made changes -
        Attachment collapse-arrays.patch [ 12469263 ]
        Todd Lipcon made changes -
        Attachment with-kvallocs.png [ 12469285 ]
        Todd Lipcon made changes -
        Attachment mslab-1.txt [ 12469336 ]
        Todd Lipcon made changes -
        Attachment icv-frag.png [ 12469345 ]
        Todd Lipcon made changes -
        Attachment mslab-2.txt [ 12469611 ]
        Todd Lipcon made changes -
        Attachment mslab-3.txt [ 12469707 ]
        Todd Lipcon made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.90.1 [ 12315548 ]
        Assignee Todd Lipcon [ tlipcon ]
        Fix Version/s 0.90.1 [ 12315548 ]
        Todd Lipcon made changes -
        Attachment mslab-4.txt [ 12469857 ]
        Todd Lipcon made changes -
        Attachment mslab-5.txt [ 12469885 ]
        Todd Lipcon made changes -
        Attachment mslab-6.txt [ 12470303 ]
        Todd Lipcon made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Release Note Adds an allocation buffer local to each memstore which is used to combat heap fragmentation. Experimental in 0.90.1 - enable by toggling hbase.hregion.memstore.mslab.enabled. Enabled by default in 0.91
        Resolution Fixed [ 1 ]
        ocean floyd made changes -
        Hadoop Flags Reviewed [ 10343 ]
        Lars Francke made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Todd Lipcon
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            19 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development