[HBASE-19024] Configurable default durability for synchronous WAL

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.1.0, 1.5.0
• Component/s: wal

Release Note

The default durability setting for the synchronous WAL is Durability.SYNC_WAL, which triggers HDFS hflush() to flush edits to the datanodes. We also support Durability.FSYNC_WAL, which instead triggers HDFS hsync() to flush _and_ fsync edits. This change introduces the new configuration setting "hbase.wal.hsync", defaulting to FALSE, that if set to TRUE changes the default durability setting for the synchronous WAL to FSYNC_WAL.

• Assignee: Harshal Jain
• Reporter: Vikas Vishwakarma
• Created: 17/Oct/17 09:26
• Updated: 17/Dec/19 14:59
• Resolved: 21/Mar/18 04:08

Description

At present we do not have an option to hsync WAL edits to the disk for better durability. In our local tests we see 10-15% latency impact of using hsync instead of hflush which is not very high.
We should have a configurable option to hysnc WAL edits instead of just sync/hflush which will call the corresponding API on the hadoop side. Currently HBase handles both SYNC_WAL and FSYNC_WAL as the same calling FSDataOutputStream sync/hflush on the hadoop side. This can be modified to let FSYNC_WAL call hsync on the hadoop side instead of sync/hflush. We can keep the default value to sync as the current behavior and hsync can be enabled based on explicit configuration.

Issue Links

• breaks: HBASE-23587 The FSYNC_WAL flag does not work on branch-2.x
• is related to: HBASE-20471 Recheck the design and implementation of FSYNC_WAL durability for WAL
• relates to: HBASE-5954 Allow proper fsync support for HBase

Sub-Tasks

• Implement fsync for AsyncFSWAL: Resolved