[HBASE-5386] [usability] Soft limit for eager region splitting of young tables

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Usability
• Assignee: Unassigned
• Reporter: Jean-Daniel Cryans
• Created: 11/Feb/12 00:18
• Updated: 12/Jun/22 20:19
• Resolved: 03/May/13 20:17

Description

Coming out of HBASE-2375, we need a new functionality much like hypertable's where we would have a lower split size for new tables and it would grow up to a certain hard limit. This helps usability in different ways:

• With that we can set the default split size much higher and users will still have good data distribution
• No more messing with force splits
• Not mandatory to pre-split your table in order to get good out of the box performance

The way Doug Judd described how it works for them, they start with a low value and then double it every time it splits. For example if we started with a soft size of 32MB and a hard size of 2GB, it wouldn't be until you have 64 regions that you hit the ceiling.

On the implementation side, we could add a new qualifier in .META. that has that soft limit. When that field doesn't exist, this feature doesn't kick in. It would be written by the region servers after a split and by the master when the table is created with 1 region.

Issue Links

• relates to: HBASE-4365 Add a decent heuristic for region size