HBase
  1. HBase
  2. HBASE-64

Add max number of mapfiles to compact at one time giveing us a minor & major compaction

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Later
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: regionserver
    • Labels:
      None

      Description

      Currently we do compaction on a region when the hbase.hstore.compactionThreshold is reached - default 3

      I thank we should configure a max number of mapfiles to compact at one time simulator to doing a minor compaction in bigtable. This keep compaction's form getting tied up in one region to long letting other regions get way to many memcache flushes making compaction take longer and longer for each region

      If we did that when a regions updates start to slack off the max number will eventuly include all mapfiles causeing a major compaction on that region. Unlike big table this would leave the master out of the process and letting the region server handle the major compaction when it has time.

      When doing a minor compaction on a few files I thank we should compact the newest mapfiles first leave the larger/older ones for when we have low updates to a region.

      1. twice.patch
        1 kB
        stack
      2. flag.patch
        2 kB
        stack
      3. flag-v2.patch
        4 kB
        stack

        Issue Links

          Activity

          Jim Kellerman made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          stack made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Later [ 7 ]
          Bryan Duxbury made changes -
          Component/s regionserver [ 12312139 ]
          Owen O'Malley made changes -
          Component/s contrib/hbase [ 12311752 ]
          Project Hadoop Core [ 12310240 ] Hadoop HBase [ 12310753 ]
          Fix Version/s 0.17.0 [ 12312913 ]
          Key HADOOP-2615 HBASE-64
          Bryan Duxbury made changes -
          Link This issue is related to HADOOP-2731 [ HADOOP-2731 ]
          stack made changes -
          Attachment flag-v2.patch [ 12374064 ]
          stack made changes -
          Attachment flag.patch [ 12374007 ]
          stack made changes -
          Attachment twice.patch [ 12374006 ]
          Jim Kellerman made changes -
          Assignee Jim Kellerman [ jimk ]
          Jim Kellerman made changes -
          Link This issue is blocked by HADOOP-2636 [ HADOOP-2636 ]
          Jim Kellerman made changes -
          Assignee Jim Kellerman [ jimk ]
          Billy Pearson made changes -
          Summary Add max number of mapfiles to compact at one time giveing up a minor & major compaction Add max number of mapfiles to compact at one time giveing us a minor & major compaction
          Billy Pearson made changes -
          Field Original Value New Value
          Component/s contrib/hbase [ 12311752 ]
          Billy Pearson created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Billy Pearson
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development