[HBASE-69] [hbase] Make cache flush triggering less simplistic

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Fix Version/s: 0.2.0
• Component/s: regionserver
• Assignee: Jim Kellerman
• Reporter: Michael Stack
• Created: 17/Jan/08 06:35
• Updated: 22/Aug/08 21:13
• Resolved: 22/Feb/08 17:17

Description

When flusher runs – its triggered when the sum of all Stores in a Region > a configurable max size – we flush all Stores though a Store memcache might have but a few bytes.

I would think Stores should only dump their memcache disk if they have some substance.

The problem becomes more acute, the more families you have in a Region.

Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size. Behavior would vary dependent on the prompt that provoked the flush. Would also log why the flush is running: optional or > max size.

This issue comes out of HADOOP-2621.

Issue Links

• blocks: HBASE-419 Move RegionServer and related classes into regionserver package
• blocks: HBASE-248 Flusher flushing back to back when it should not be
• blocks: HBASE-395 [hbase] Refactor HStore and internal (and related) classes into hbase.store subpackage
• blocks: HBASE-64 Add max number of mapfiles to compact at one time giveing us a minor & major compaction
• blocks: HBASE-75 [hbase] Refactor classes into Client, Master, and Regionserver packages
• is related to: HBASE-192 Memcache flush flushing every 60 secs with out considering the max memcache size