Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-6466

Enable multi-thread for memstore flush

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.95.2
    • Fix Version/s: 0.95.0
    • Component/s: regionserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      If the KV is large or Hlog is closed with high-pressure putting, we found memstore is often above the high water mark and block the putting.

      So should we enable multi-thread for Memstore Flush?

      Some performance test data for reference,

      1.test environment :
      random writting;upper memstore limit 5.6GB;lower memstore limit 4.8GB;400 regions per regionserver;row len=50 bytes, value len=1024 bytes;5 regionserver, 300 ipc handler per regionserver;5 client, 50 thread handler per client for writing

      2.test results:

      one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per regionserver, appears many aboveGlobalMemstoreLimit blocking

      two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per regionserver,

      200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per regionserver, Flush:18.6MB/s per regionserver

        Attachments

        1. 6466-v6.patch
          22 kB
          Ted Yu
        2. 6466-v7.patch
          16 kB
          Chunhui Shen
        3. HBASE-6466.patch
          12 kB
          Chunhui Shen
        4. HBASE-6466v2.patch
          12 kB
          Chunhui Shen
        5. HBASE-6466v3.1.patch
          26 kB
          Sergey Shelukhin
        6. HBASE-6466v3.patch
          15 kB
          Chunhui Shen
        7. HBASE-6466-v4.patch
          28 kB
          Sergey Shelukhin
        8. HBASE-6466-v4.patch
          28 kB
          Sergey Shelukhin
        9. HBASE-6466-v5.patch
          24 kB
          Sergey Shelukhin

          Issue Links

            Activity

              People

              • Assignee:
                zjushch Chunhui Shen
                Reporter:
                zjushch Chunhui Shen
              • Votes:
                0 Vote for this issue
                Watchers:
                24 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: