[HBASE-7849] Provide administrative limits around bulkloads of files into a single region

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.98.0, 0.98.1, 0.99.0
• Component/s: regionserver
• Hadoop Flags: Reviewed
• Assignee: Jimmy Xiang
• Reporter: Harsh J
• Created: 14/Feb/13 22:36
• Updated: 21/Feb/15 23:32
• Resolved: 12/Feb/14 19:56

Description

Given the current mechanism, it is possible for users to flood a single region with 1k+ store files via the bulkload API and basically cause the region to become a flying dutchman - never getting assigned successfully again.

Ideally, an administrative limit could solve this. If the bulkload RPC call can check if the region already has X store files, then it can reject the request to add another and throw a failure at the client with an appropriate message.

This may be an intrusive change, but seems necessary in perfecting the gap between devs and ops in managing a HBase clusters. This would especially prevent abuse in form of unaware devs not pre-splitting tables before bulkloading things in. Currently, this leads to ops pain, as the devs think HBase has gone non-functional and begin complaining.

Issue Links

• is related to: HBASE-7850 Bulkload final step can detect and pre-split tables