[HBASE-26579] Set storage policy of recovered edits when wal storage type is configured

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.5.0, 3.0.0-alpha-2, 2.4.10
• Component/s: Recovery
• Hadoop Flags: Reviewed
• Assignee: zhuobin zheng
• Reporter: zhuobin zheng
• Created: 15/Dec/21 10:40
• Updated: 18/Jan/22 15:47
• Resolved: 18/Dec/21 15:37

Description

In our cluster, we has many SSD and a little HDD.  (Most table configured storage policy ONE_SSD, and all wals is configured ALL_SSD)

when all cluster down, It's difficult to recovery cluster. Because HDD Disk IO bottleneck (Almost all disk io util is 100%).

I think the most hdfs operation when recovery is split wal to recovered edits dir, And read it.

And it goes better when i stop hbase and set all recovered.edits to ALL_SSD.

So we can get benifit of recovery time if we set recovered.edits dir to better storage like WAL.

Now i reuse config item  hbase.wal.storage.policy to set recovered.edits storage type. Because I did not find a scenario where they use different storage Policy

Issue Links

• is a parent of: HBASE-26678 Backport HBASE-26579 to branch-1
• links to: GitHub Pull Request #3948
• links to: GitHub Pull Request #4037