[HBASE-10278] Provide better write predictability

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: wal
• Assignee: Unassigned
• Reporter: Himanshu Vashishtha
• Created: 04/Jan/14 00:58
• Updated: 14/Sep/23 06:00
• Resolved: 17/Jun/22 15:36

Description

Currently, HBase has one WAL per region server.
Whenever there is any latency in the write pipeline (due to whatever reasons such as n/w blip, a node in the pipeline having a bad disk, etc), the overall write latency suffers.

Jonathan Hsieh and I analyzed various approaches to tackle this issue. We also looked at HBASE-5699, which talks about adding concurrent multi WALs. Along with performance numbers, we also focussed on design simplicity, minimum impact on MTTR & Replication, and compatibility with 0.96 and 0.98. Considering all these parameters, we propose a new HLog implementation with WAL Switching functionality.

Please find attached the design doc for the same. It introduces the WAL Switching feature, and experiments/results of a prototype implementation, showing the benefits of this feature.
The second goal of this work is to serve as a building block for concurrent multiple WALs feature.

Please review the doc.

Issue Links

• is related to: HBASE-11283 [0.89-fb] Roll HLog if sync time is anomalous
• is related to: HBASE-5699 Run with > 1 WAL in HRegionServer

Sub-Tasks

• Divide HLog interface into User and Implementor specific interfaces: Closed
• Implement 0.89fb-style "roll on slow sync": Closed