[HBASE-10713] A MemStore implementation with in memory flushes to CellBlocks

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: Anoop Sam John
• Created: 10/Mar/14 16:52
• Updated: 12/Jun/22 19:00
• Resolved: 21/Oct/16 06:22

Description

After HBASE-10648 we can plugin any implementation for MemStore. This issue aims at coming up with an implementation, in which we will have in between in memory flushes. This will reduce the need to keep lots of KVs in heap as well as in CSLM. CSLM perform poor when no# items in it increases. We can keep create CellBlocks (contigous byte[] like HFile block) out of KVs and keep it as one object rather than many KVs. At some point in time, MemStore might have N CellBlocks and one CSLM.

These in memory CellBlocks can be compacted to one bigger block in between. We can target that in follow on tasks once the basic code is ready.

Issue Links

• is blocked by: HBASE-11401 Late-binding sequenceid presumes a particular KeyValue mvcc format hampering experiment
• is related to: HBASE-10191 Move large arena storage off heap
• relates to: HBASE-13408 HBase In-Memory Memstore Compaction
• relates to: HBASE-3993 Alternatives to ConcurrentSkipListMap in MemStore
• relates to: HBASE-14918 In-Memory MemStore Flush and Compaction
• requires: HBASE-10648 Pluggable Memstore
• requires: HBASE-10771 Primitive type put/get APIs in ByteRange