[HBASE-3443] ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.90.0, 0.90.1, 0.90.2, 0.90.3, 0.90.4, 0.90.5, 0.90.6, 0.92.0, 0.92.1
• Fix Version/s: 0.94.0, 0.95.0
• Component/s: regionserver
• Labels: corruption
• Hadoop Flags: Reviewed

Release Note

This is a correctness fix and will incur a 10-20% performance penalty for ICV and Increment operations. Other operations are not affected.

• Assignee: Lars Hofhansl
• Reporter: Kannan Muthukkaruppan
• Created: 13/Jan/11 23:18
• Updated: 18/Sep/13 22:13
• Resolved: 12/Apr/12 17:46

Description

For incrementColumnValue() HBASE-3082 adds an optimization to check memstores first, and only if not present in the memstore then check the store files. In the presence of deletes, the above optimization is not reliable.

If the column is marked as deleted in the memstore, one should not look further into the store files. But currently, the code does so.

Sample test code outline:

admin.createTable(desc)

table = HTable.new(conf, tableName)

table.incrementColumnValue(Bytes.toBytes("row"), cf1name, Bytes.toBytes("column"), 5);

admin.flush(tableName)
sleep(2)

del = Delete.new(Bytes.toBytes("row"))
table.delete(del)

table.incrementColumnValue(Bytes.toBytes("row"), cf1name, Bytes.toBytes("column"), 5);

get = Get.new(Bytes.toBytes("row"))
keyValues = table.get(get).raw()
keyValues.each do |keyValue|
  puts "Expect 5; Got Value=#{Bytes.toLong(keyValue.getValue())}";
end

The above prints:

Expect 5; Got Value=10

Issue Links

• is related to: HBASE-4536 Allow CF to retain deleted rows
• is related to: HBASE-3082 For ICV gets, first look in MemStore before reading StoreFiles