[HBASE-20952] Re-visit the WAL API

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Component/s: wal
• Assignee: Unassigned
• Reporter: Josh Elser
• Created: 26/Jul/18 17:22
• Updated: 11/Jun/22 18:20
• Resolved: 11/Jun/22 18:20

Description

Take a step back from the current WAL implementations and think about what an HBase WAL API should look like. What are the primitive calls that we require to guarantee durability of writes with a high degree of performance?

The API needs to take the current implementations into consideration. We should also have a mind for what is happening in the Ratis LogService (but the LogService should not dictate what HBase's WAL API looks like RATIS-272).

Other "systems" inside of HBase that use WALs are replication and backup&restore. Replication has the use-case for "tail"'ing the WAL which we should provide via our new API. B&R doesn't do anything fancy (IIRC). We should make sure all consumers are generally going to be OK with the API we create.

The API may be "OK" (or OK in a part). We need to also consider other methods which were "bolted" on such as AbstractFSWAL and WALFileLengthProvider. Other corners of "WAL use" (like the WALSplitter should also be looked at to use WAL-APIs only).

We also need to make sure that adequate interface audience and stability annotations are chosen.

Issue Links

• contains: HBASE-21020 Determine WAL API changes for replication
• Is contained by: HBASE-20951 Ratis LogService backed WALs
• relates to: RATIS-272 LogService: Design ideal API
• relates to: HBASE-26890 Make the WAL interface async so sync replication can be built up on the WAL interface
• links to: Condensed (API Only) ReviewBoard
• links to: Decouple WAL and Replication
• links to: Design Doc

Sub-Tasks

• Introduce WALIdentity interface: Resolved
• Disable nightly builds on snoozing branch HBASE-20952: Resolved