[HBASE-13082] Coarsen StoreScanner locks to RegionScanner

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0, 2.0.0
• Component/s: Performance, Scanners
• Hadoop Flags: Reviewed

Release Note

After this JIRA we will not be doing any scanner reset after compaction during a course of a scan. The files that were compacted will still be continued to be used in the scan process. The compacted files will be archived by a background thread that runs every 2 mins by default only when there are no active scanners on those comapcted files. The above duration can be controlled using the knob 'hbase.hfile.compactions.cleaner.interval'.

• Assignee: ramkrishna.s.vasudevan
• Reporter: Lars Hofhansl
• Created: 21/Feb/15 06:36
• Updated: 23/Jun/22 18:44
• Resolved: 22/Jan/16 18:44

Description

Continuing where HBASE-10015 left of.
We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock already held by the RegionScanner.
In tests this shows quite a scan improvement and reduced CPU (the fences make the cores wait for memory fetches).

There are some drawbacks too:

• All calls to RegionScanner need to be remain synchronized
• Implementors of coprocessors need to be diligent in following the locking contract. For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation (not picking on Phoenix, this one is my fault as I told them it's OK)
• possible starving of flushes and compaction with heavy read load. RegionScanner operations would keep getting the locks and the flushes/compactions would not be able finalize the set of files.

I'll have a patch soon.

Issue Links

• is related to: HBASE-20704 Sometimes some compacted storefiles are not archived on region close
• relates to: HBASE-18186 Frequent FileNotFoundExceptions in region server logs
• relates to: HBASE-14970 Backport HBASE-13082 and its sub-jira to branch-1
• relates to: HBASE-10060 Unsynchronized scanning
• relates to: HBASE-13071 Hbase Streaming Scan Feature
• relates to: HBASE-13090 Progress heartbeats for long running scanners
• links to: RB link

Sub-Tasks

• Seek only to the newly flushed file on scanner reset on flush: Closed