[HBASE-20196] Maintain all regions with same size in memstore flusher

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1
• Hadoop Flags: Reviewed
• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 14/Mar/18 15:06
• Updated: 28/Mar/18 02:51
• Resolved: 19/Mar/18 15:11

Description

Here is the javadoc for getCopyOfOnlineRegionsSortedByOffHeapSize() :

   *   the biggest.  If two regions are the same size, then the last one found wins; i.e. this
   *   method may NOT return all regions.

Currently value type is HRegion - we only store one region per size.
I think we should change value type to Collection<HRegion> so that we don't miss any region (potentially with big size).

e.g. Suppose there are there regions (R1, R2 and R3) with sizes 100, 100 and 1, respectively.
Using the current data structure, R2 would be stored in the Map, evicting R1 from the Map.
This means that the current code would choose to flush regions R2 and R3, releasing 101 from memory.
If value type is changed to Collection<HRegion>, we would flush both R1 and R2. This achieves faster memory reclamation.

Confirmed with eshcar over in HBASE-20090