[HBASE-14457] Umbrella: Improve Multiple WAL for production usage

• Type: Umbrella
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0, 2.0.0
• Assignee: Yu Li
• Reporter: Yu Li
• Created: 21/Sep/15 15:11
• Updated: 01/Jul/22 21:35
• Resolved: 17/Jan/16 11:33

Description

HBASE-5699 proposed the idea to run with multiple WAL in regionserver and did a great initial work there, but when trying to use it in our production cluster, we still found several issues to resolve, like tracking multiple WAL paths in replication (HBASE-6617), fixing UT with multiwal provider (HBASE-14411), introducing a namespace-based strategy for RegionGroupingProvider (HBASE-14456), etc. This is an umbrella including(but not limited of) all these works and efforts to make multiple wal ready for production usage and give user a clear picture about it.

Besides the developing works done, I'd also like to share some scenarios and testing/online data in this JIRA about our usage/performance of multiple wal, to(hopefully) help people better judge whether to enable multiple wal or not in their own cluster and what they might gain.

Issue Links

• contains: HBASE-14411 Fix unit test failures when using multiwal as default WAL provider
• contains: HBASE-14456 Implement a namespace-based region grouping strategy for RegionGroupingProvider
• contains: HBASE-6617 ReplicationSourceManager should be able to track multiple WAL paths
• contains: HBASE-14306 Refine RegionGroupingProvider: fix issues and make it more scalable
• contains: HBASE-14448 Refine RegionGroupingProvider Phase-2: remove provider nesting and formalize wal group name
• relates to: HBASE-15093 Replication can report incorrect size of log queue for the global source when multiwal is enabled
• relates to: HBASE-15131 Turn on multi-WAL by default
• relates to: HBASE-5699 Run with > 1 WAL in HRegionServer