[HBASE-10500] Some tools OOM when BucketCache is enabled

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.98.0, 0.96.0, 0.99.0
• Fix Version/s: 0.96.2, 0.98.1, 0.99.0
• Component/s: HFile
• Assignee: Nick Dimiduk
• Reporter: Nick Dimiduk
• Created: 11/Feb/14 18:05
• Updated: 21/Feb/15 23:29
• Resolved: 12/Feb/14 17:16

Description

Running hbck --repair or LoadIncrementalHFiles when BucketCache is enabled in offheap mode can cause OOM. This is apparently because bin/hbase does not include $HBASE_REGIONSERVER_OPTS for these tools. This results in HRegion or HFileReaders initialized with a CacheConfig that doesn't have the necessary Direct Memory.

Possible solutions include:

• disable blockcache in the config used by hbck when running its repairs
• include HBASE_REGIONSERVER_OPTS in the HBaseFSCK startup arguments

I'm leaning toward the former because it's possible that hbck is run on a host with different hardware profile as the RS.