HBase
  1. HBase
  2. HBASE-834

'Major' compactions and upper bound on files we compact at any one time

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.2.1, 0.18.0
    • Fix Version/s: 0.2.1, 0.18.0
    • Component/s: None
    • Labels:
      None

      Description

      From Billy in HBASE-64, which we closed because it got pulled all over the place:

      Currently we do compaction on a region when the hbase.hstore.compactionThreshold is reached - default 3
      
      I thank we should configure a max number of mapfiles to compact at one time simulator to doing a minor compaction in bigtable. This keep compaction's form getting tied up in one region to long letting other regions get way to many memcache flushes making compaction take longer and longer for each region
      
      If we did that when a regions updates start to slack off the max number will eventuly include all mapfiles causeing a major compaction on that region. Unlike big table this would leave the master out of the process and letting the region server handle the major compaction when it has time.
      
      When doing a minor compaction on a few files I thank we should compact the newest mapfiles first leave the larger/older ones for when we have low updates to a region.
      
      1. 834.patchv4-trunk.txt
        1.0 kB
        Billy Pearson
      2. 834-0.2.1-patchv4.txt
        0.9 kB
        Billy Pearson
      3. 834-0.2.1-patchv3.txt
        8 kB
        Billy Pearson
      4. 834-0.2.1-patchv2.txt
        5 kB
        Billy Pearson
      5. 834-0.2.1-patch.txt
        5 kB
        Billy Pearson
      6. 834-patch.txt
        5 kB
        Billy Pearson

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Billy Pearson
            Reporter:
            stack
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development