Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-6371

[89-fb] Tier based compaction

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Done
    • None
    • None
    • None
    • beginner

    Description

      Currently, the compaction selection is not very flexible and is not sensitive to the hotness of the data. Very old data is likely to be accessed less, and very recent data is likely to be in the block cache. Both of these considerations make it inefficient to compact these files as aggressively as other files. In some use-cases, the access-pattern is particularly obvious even though there is no way to control the compaction algorithm in those cases.

      In the new compaction selection algorithm, we plan to divide the candidate files into different levels according to oldness of the data that is present in those files. For each level, parameters like compaction ratio, minimum number of store-files in each compaction may be different. Number of levels, time-ranges, and parameters for each level will be configurable online on a per-column family basis.

      Attachments

        1. HBASE-6371-089fb-commit.patch
          148 kB
          Sergey Shelukhin
        2. HBase_Tier_Base_Compaction.pdf
          1.05 MB
          Liyin Tang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            liyin Liyin Tang
            akashnil07 Akashnil
            Votes:
            0 Vote for this issue
            Watchers:
            36 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment