Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13259

mmap() based BucketCache IOEngine

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.98.10
    • Fix Version/s: 2.0.0
    • Component/s: BlockCache
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      mmap() based bucket cache can be configured by specifying the property
      {code}
      <property>
        <name>hbase.bucketcache.ioengine</name>
        <value> mmap://filepath </value>
      </property>
      {code}
      This mode of bucket cache is ideal when your file based bucket cache size is lesser than then available RAM. When the cache is bigger than the available RAM then the kernel page faults will make this cache perform lesser particularly in case of scans.
      Show
      mmap() based bucket cache can be configured by specifying the property {code} <property>   <name>hbase.bucketcache.ioengine</name>   <value> mmap://filepath </value> </property> {code} This mode of bucket cache is ideal when your file based bucket cache size is lesser than then available RAM. When the cache is bigger than the available RAM then the kernel page faults will make this cache perform lesser particularly in case of scans.

      Description

      Of the existing BucketCache IOEngines, FileIOEngine uses pread() to copy data from kernel space to user space. This is a good choice when the total working set size is much bigger than the available RAM and the latency is dominated by IO access. However, when the entire working set is small enough to fit in the RAM, using mmap() (and subsequent memcpy()) to move data from kernel space to user space is faster. I have run some short keyval gets tests and the results indicate a reduction of 2%-7% of kernel CPU on my system, depending on the load. On the gets, the latency histograms from mmap() are identical to those from pread(), but peak throughput is close to 40% higher.

      This patch modifies ByteByfferArray to allow it to specify a backing file.

      Example for using this feature: set hbase.bucketcache.ioengine to mmap:/dev/shm/bucketcache.0 in hbase-site.xml.

      Attached perf measured CPU usage breakdown in flames graph.

        Attachments

        1. HBASE-13259_v3.patch
          19 kB
          ramkrishna.s.vasudevan
        2. HBASE-13259_v4.patch
          18 kB
          ramkrishna.s.vasudevan
        3. HBASE-13259_v5.patch
          18 kB
          ramkrishna.s.vasudevan
        4. HBASE-13259_v6.patch
          19 kB
          ramkrishna.s.vasudevan
        5. HBASE-13259.patch
          13 kB
          Zee Chen
        6. HBASE-13259-v2.patch
          13 kB
          Zee Chen
        7. ioread-1.svg
          1.37 MB
          Zee Chen
        8. mmap-0.98-v1.patch
          12 kB
          Zee Chen
        9. mmap-1.svg
          2.05 MB
          Zee Chen
        10. mmap-trunk-v1.patch
          12 kB
          Zee Chen

          Activity

            People

            • Assignee:
              zeocio Zee Chen
              Reporter:
              zeocio Zee Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: