Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-9857

Blockcache prefetch option

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 0.99.0, 0.98.3
    • None
    • None
    • Reviewed

    Description

      Attached patch implements a prefetching function for HFile (v3) blocks, if indicated by a column family or regionserver property. The purpose of this change is to as rapidly after region open as reasonable warm the blockcache with all the data and index blocks of (presumably also in-memory) table data, without counting those block loads as cache misses. Great for fast reads and keeping the cache hit ratio high. Can tune the IO impact versus time until all data blocks are in cache. Works a bit like CompactSplitThread. Makes some effort not to stampede.

      I have been using this for setting up various experiments and thought I'd polish it up a bit and throw it out there. If the data to be preloaded will not fit in blockcache, or if as a percentage of blockcache it is large, this is not a good idea, will just blow out the cache and trigger a lot of useless GC activity. Might be useful as an expert tuning option though. Or not.

      Attachments

        1. 9857.patch
          61 kB
          Andrew Kyle Purtell
        2. 9857.patch
          61 kB
          Andrew Kyle Purtell
        3. HBASE-9857-0.98.patch
          64 kB
          Andrew Kyle Purtell
        4. HBASE-9857-trunk.patch
          63 kB
          Andrew Kyle Purtell
        5. HBASE-9857-trunk.patch
          66 kB
          Andrew Kyle Purtell

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            apurtell Andrew Kyle Purtell
            apurtell Andrew Kyle Purtell
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment