[HADOOP-18671] Add recoverLease(), setSafeMode(), isFileClosed() APIs to FileSystem

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.6
• Fix Version/s: 3.4.0, 3.2.5, 3.3.6
• Component/s: fs
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.6
• Hadoop Flags: Reviewed
• Assignee: Tak-Lon (Stephen) Wu
• Reporter: Wei-Chiu Chuang
• Created: 20/Mar/23 23:23
• Updated: 27/Jan/24 08:33
• Resolved: 03/May/23 10:06

Description

We are in the midst of enabling HBase and Solr to run on Ozone.

An obstacle is that HBase relies heavily on HDFS APIs and semantics for its Write Ahead Log (WAL) file (similarly, for Solr's transaction log). We propose to push up these HDFS APIs, i.e. recoverLease(), setSafeMode(), isFileClosed() to FileSystem abstraction so that HBase and other applications do not need to take on Ozone dependency at compile time. This work will (hopefully) enable HBase to run on other storage system implementations in the future.

There are other HDFS features that HBase uses, including hedged read and favored nodes. Those are FS-specific optimizations and are not critical to enable HBase on Ozone.

Issue Links

• is required by: HDDS-8436 Support setSafeMode(), isFileClosed() FileSystem API
• is required by: HBASE-27740 Support Ozone as a WAL backing storage
• is required by: HBASE-27769 Use hasPathCapability to support recoverLease, setSafeMode, isFileClosed for non-HDFS file system
• links to: GitHub Pull Request #5553
• links to: GitHub Pull Request #5619
• links to: GitHub Pull Request #5620

Sub-Tasks

• Backport HADOOP-18671 to branch-3.3: Resolved
• Backport HADOOP-18671 to branch-3.2: Resolved
• Backport HADOOP-18671 to branch-2.10.x: Resolved