Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21879 Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
  3. HBASE-22422

Retain an ByteBuff with refCnt=0 when getBlock from LRUCache

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0, 2.3.0
    • Component/s: BlockCache
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      After runing YCSB scan/get benchmark in our XiaoMi cluster, we found the get QPS dropped from 25000/s to hunderds per second in a cluster with five nodes.

      After enable the debug log at YCSB client side, I found the following stacktrace , see https://issues.apache.org/jira/secure/attachment/12968745/image-2019-05-15-12-00-03-641.png.

      After looking into the stractrace, I can ensure that the zero refCnt block is an intermedia index block, see [2] http://hbase.apache.org/images/hfilev2.png

      Need a patch to fix this.

        Attachments

        1. 0001-debug2.patch
          4 kB
          Zheng Hu
        2. 0001-debug2.patch
          3 kB
          Zheng Hu
        3. 0001-debug2.patch
          3 kB
          Zheng Hu
        4. 0001-debug3.patch
          4 kB
          Zheng Hu
        5. 0001-debug4.patch
          23 kB
          Zheng Hu
        6. debug.patch
          4 kB
          Zheng Hu
        7. failed-to-check-positive-on-web-ui.png
          304 kB
          Zheng Hu
        8. HBASE-22422.HBASE-21879.v01.patch
          13 kB
          Zheng Hu
        9. HBASE-22422.HBASE-21879.v02.patch
          30 kB
          Zheng Hu
        10. HBASE-22422-qps-after-fix-the-zero-retain-bug.png
          386 kB
          Zheng Hu
        11. image-2019-05-15-12-00-03-641.png
          383 kB
          Zheng Hu
        12. LRUBlockCache-getBlock.png
          158 kB
          Zheng Hu

          Issue Links

          links to

          Web Link GitHub Pull Request #242

            Activity

              People

              • Assignee:
                openinx Zheng Hu
                Reporter:
                openinx Zheng Hu
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: