[HBASE-12848] Utilize Flash storage for WAL

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.0, 2.0.0
• Hadoop Flags: Reviewed

Release Note

For users on a version of Hadoop that supports tiered storage policies (i.e. Apache Hadoop 2.6.0+), HBase now allows users to opt-in to having the write ahead log placed on the SSD tier. Users on earlier versions of Hadoop will be unable to take advantage of this feature.

Use of tiered storage is controlled by a new RegionServer config, hbase.wal.storage.policy. It defaults to the value 'NONE', which will rely on HDFS defaults for a policy decision.

User can specify ONE_SSD or ALL_SSD as the value:
ONE_SSD: place only one replica of WAL files in SSD and the remaining in default storage
ALL_SSD: all replica for WAL files are placed on SSD

See [the HDFS docs on storage policy|http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html]

• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 13/Jan/15 17:38
• Updated: 06/Jun/18 19:14
• Resolved: 15/Jan/15 17:06

Description

One way to improve data ingestion rate is to make use of Flash storage.
HDFS is doing the heavy lifting - see HDFS-7228.

We assume an environment where:
1. Some servers have a mix of flash, e.g. 2 flash drives and 4 traditional drives.
2. Some servers have all traditional storage.
3. RegionServers are deployed on both profiles within one HBase cluster.

This JIRA allows WAL to be managed on flash in a mixed-profile environment.

Issue Links

• relates to: HBASE-18118 Default storage policy if not configured cannot be "NONE"
• relates to: HBASE-20691 Storage policy should allow deferring to HDFS