HBase
  1. HBase
  2. HBASE-5369

Compaction selection based on the hotness of the HFile's block in the block cache

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      HBase reserves a large set memory for the block cache and the cached blocks will be age out in a LRU fashion. Obviously, we don't want to age out the blocks which are still hot. However, when the compactions are starting, these hot blocks may naturally be invalid. Considering that the block cache has already known which HFiles these hot blocks come from, the compaction selection algorithm could just simply skip compact these HFiles until these block cache become cold.

      For example, if there is a HFile and 80% of blocks for this HFile is be cached, which means this HFile is really hot, then just skip this HFile during the compaction selection.
      The percentage of hot blocks should be configured as a high bar to make sure that HBase are still making progress for the compaction.

        Issue Links

          Activity

          There are no comments yet on this issue.

            People

            • Assignee:
              Liyin Tang
              Reporter:
              Liyin Tang
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:

                Development