[HBASE-4241] Optimize flushing of the Store cache for max versions and (new) min versions

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.92.0
• Fix Version/s: 0.92.0
• Component/s: regionserver
• Hadoop Flags: Reviewed
• Assignee: Lars Hofhansl
• Reporter: Lars Hofhansl
• Created: 23/Aug/11 04:35
• Updated: 20/Nov/15 11:55
• Resolved: 25/Aug/11 16:36

Description

As discussed with with Jon, there is room for improvement in how the memstore is flushed to disk.
Currently only expired KVs are pruned before flushing, but we can also prune versions if we find at least maxVersions versions in the memstore.
The same holds for the new minversion feature: If we find at least minVersion versions in the store we can remove all further versions that are expired.

Generally we should use the same mechanism here that is used for Compaction. I.e. StoreScanner. We only need to add a scanner to Memstore that can scan along the current snapshot.

Issue Links

• relates to: HBASE-10447 Memstore flusher scans storefiles also when the scanner heap gets reset

Sub-Tasks

• Store could miss rows during flush: Closed