Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21879 Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
  3. HBASE-22387

Evaluate the get/scan performance after reading HFile block into offheap directly

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • None
    • None
    • None

    Description

      Now, all sub-tasks has been resolved now (except the HBASE-21946 because of the hadoop dependency problem), will provide some performance benckmarks to show the latency improvement.

      Attachments

        1. async-prof-pid-24534-alloc-2.svg
          412 kB
          Zheng Hu
        2. 0001-Initialize-a-small-buffer-to-send-the-short-circuit-.patch
          2 kB
          Zheng Hu
        3. screenshot-2.png
          111 kB
          Zheng Hu
        4. async-prof-pid-25042-cpu-1.svg
          441 kB
          Zheng Hu
        5. async-prof-pid-25042-alloc-2.svg
          373 kB
          Zheng Hu
        6. async-prof-pid-25042-lock-3.svg
          72 kB
          Zheng Hu
        7. G1GC-stage-stw.case02-with-buffer-size-64KB.png
          543 kB
          Zheng Hu
        8. G1GC-stw.case02-with-buffer-size-64KB.png
          516 kB
          Zheng Hu
        9. QPS-latency.case02-with-buffer-size-64KB.png
          391 kB
          Zheng Hu
        10. QPS-latency.case01.png
          331 kB
          Zheng Hu
        11. test-cluster-configuration-details.png
          98 kB
          Zheng Hu
        12. G1GC-stage-stw.case01.png
          645 kB
          Zheng Hu
        13. G1GC-stw.case01.png
          692 kB
          Zheng Hu
        14. blockReaderRemote.png
          272 kB
          Zheng Hu
        15. blocksByHFile-stack-trace.png
          189 kB
          Zheng Hu
        16. screenshot-1.png
          46 kB
          Zheng Hu

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HBASE-22504 Optimize the MultiByteBuff#get(ByteBuffer, offset, len)

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-14535 The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-14541 When evictableMmapped or evictable size is zero, do not throw NoSuchElementException

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HBASE-22425 Balance shell command broken in HBase-3.0.0

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-14532 [book] dfs.client.read.shortcircuit is referenced as hbase-site.xml config and not described in section 7

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              openinx Zheng Hu
              openinx Zheng Hu
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: