[HBASE-6508] [0.89-fb] Filter out edits at log split time

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.89-fb
• Fix Version/s: 0.89-fb
• Component/s: master, regionserver, wal
• Assignee: Alex Feinberg
• Reporter: Alex Feinberg
• Created: 03/Aug/12 19:22
• Updated: 13/Jun/22 18:59
• Resolved: 08/Aug/12 21:21

Description

At log splitting time, we can filter out many edits if we have a conservative estimate of what was saved last in each region.

This patch does the following:

1) When a region server flushes a MemStore to HFile, store the last flushed sequence id for the region in a map.

2) Send the map to master it as a part of the region server report.

3) Adds an RPC call in HMasterRegionInterface to allow a region server to query the last last flushed sequence id for a region.

4) Skips any log entry with sequence id lower than last flushed sequence id for the region during log split time.

5) When a region is removed from a region server, removed the the entry for that region from the map, so that it isn't sent during the next report.

This can reduce downtime when a regionserver goes down quite a bit.

Issue Links

• is related to: HBASE-9568 backport HBASE-6508 to 0.94
• relates to: HBASE-5843 Improve HBase MTTR - Mean Time To Recover
• links to: Phabricator D4509