[HBASE-3483] No soft flush trigger on global memstore limit

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.90.0
• Fix Version/s: 0.90.1
• Component/s: Performance, regionserver
• Hadoop Flags: Reviewed
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 26/Jan/11 23:42
• Updated: 20/Nov/15 12:41
• Resolved: 01/Feb/11 06:22

Description

I think this is the reason people see long blocking periods under write load.

Currently when we hit the global memstore limit, we call reclaimMemStoreMemory() which is synchronized - thus everyone has to wait until the memory has flushed down to the low water mark. This causes every writer to block for 10-15 seconds on a large heap.

Instead we should start triggering flushes (in another thread) whenever we're above the low water mark. Then only block writers when we're above the high water mark.

Issue Links

• duplicates: HBASE-2158 Change how high/low global limit works; start taking on writes as soon as we dip below high limit rather than block until low limit as we currently do.
• is related to: HBASE-3438 Cluster Wide Pauses