[HBASE-25382] Super Split Meta

• Type: Umbrella
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: meta
• Assignee: Unassigned
• Reporter: Michael Stack
• Created: 10/Dec/20 20:22
• Updated: 10/Nov/23 06:17

Description

Currently, the hbase:meta Table is made of one Region only. It is not splittable. We would like to make it so the hbase:meta table can be split just as we for user-space tables as they grow in size.

Why split the hbase:meta table?

A single Region hbase:meta Table hotspots as cluster size grows. At extreme, hotspotting brings on special-casing with deploys dedicating nodes solely to the hosting of the single hbase:meta Region to better carry the higher load (but also to isolate hbase:meta if heavily trafficked neighbor Regions). Splitting, currently disallowed, will enable distributing the hbase:meta Table Regions, and thereby load, across the cluster. A splittable hbase:meta table will also alleviate concerns enlarging the hbase:meta table whether by adding more meta data per Region entry – e.g. keeping the list of Region HFiles in the hbase:meta table – or running with more, smaller Regions rather than a few large Regions.

This is not the first issue to concern itself with meta splitting (HBASE-11288, HBASE-24950). By aggreement. – see the base of HBASE-11288 – this issue supplants all previous JIRAs and design efforts. It is a reset. We start by listing requirements in the attached Super Split Meta Design.

Issue Links

• incorporates: HBASE-11288 Splittable Meta
• incorporates: HBASE-24950 Another solution for Splittable Meta
• is related to: HBASE-26193 Do not store meta region location as permanent state on zookeeper
• relates to: HBASE-28194 New Splittable Meta
• links to: Super Split Meta Design

Sub-Tasks

• POC: hbase:meta,,1 as ROOT: Open