[HBASE-20951] Ratis LogService backed WALs

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Later
• Component/s: wal
• Assignee: Unassigned
• Reporter: Josh Elser
• Created: 26/Jul/18 17:13
• Updated: 13/Jun/22 19:52
• Resolved: 13/Jun/22 19:52

Description

Umbrella issue for the Ratis+WAL work:

Design doc: https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit#

The (over-simplified) goal is to re-think the current WAL APIs we have now, ensure that they are de-coupled from the notion of being backed by HDFS, swap the current implementations over to the new API, and then wire up the Ratis LogService to the new WAL API.

Issue Links

• contains: HBASE-20952 Re-visit the WAL API
• is related to: RATIS-271 Ratis-backed distributed log: "LogService"

Sub-Tasks

• Write to Ratis LogService as a WAL: Resolved
• Support reading from a LogService WAL: Resolved
• Port FSHLog to new WAL API: Resolved
• Port AsyncFSWAL over to the new WAL API: Resolved
• Switch replication over to use the new WAL API: Resolved
• Port WAL recovery code to use the new WAL API: Resolved
• Port backup and restore to new WAL API: Resolved
• Recover RAFT quorum membership loss: Resolved
• Expose Ratis LogService metrics: Resolved
• LogStream Metadata Tracking: Resolved
• Benchmark LogService WALs: Resolved
• Determine WAL API changes for replication: Resolved
• Investigate sync-replication: Resolved
• Make WALFactory only used for creating WALProviders: Resolved