Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-23066

Allow cache on write during compactions when prefetching is enabled

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.4.10
    • Fix Version/s: 2.3.0, 1.6.0
    • Component/s: Compaction, regionserver
    • Labels:
      None

      Description

      In cases where users care a lot about read performance for tables that are small enough to fit into a cache (or the cache is large enough), prefetchOnOpen can be enabled to make the entire table available in cache after the initial region opening is completed. Any new data can also be guaranteed to be in cache with the cacheBlocksOnWrite setting.

      However, the missing piece is when all blocks are evicted after a compaction. We found very poor performance after compactions for tables under heavy read load and a slower backing filesystem (S3). After a compaction the prefetching threads need to compete with threads servicing read requests and get constantly blocked as a result. 

      This is a proposal to introduce a new cache configuration option that would cache blocks on write during compaction for any column family that has prefetch enabled. This would virtually guarantee all blocks are kept in cache after the initial prefetch on open is completed allowing for guaranteed steady read performance despite a slow backing file system.

        Attachments

        1. prefetchCompactedBlocksOnWrite.patch
          12 kB
          Jacob LeBlanc
        2. HBASE-23066.patch
          10 kB
          Jacob LeBlanc
        3. performance_results.png
          28 kB
          Jacob LeBlanc

          Issue Links

            Activity

              People

              • Assignee:
                jacob.leblanc Jacob LeBlanc
                Reporter:
                jacob.leblanc Jacob LeBlanc
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated: