After reading the proposal on HBASE-10278, I realized there are many ways to make the Accumulo WAL roll-over faster.
- Open two WALogs, but use only one until it reaches the WALog roll-over size
- Rollover consists only of swapping the writers
- WALog roll consists of the final close, which can happen in parallel
- Don't mark the tablets with log entries: they are already marked with the tserver
- The tserver can make notes about the logs-in-use in the metadata table(s) as part of opening the log.
- The master can copy the log entries to tablets while unassigning them, piggybacking on the unassigment mutation.
- Tablet servers can remove their current log entries from the metadata tables when they have no tablets using them.
There are two issues:
- tablets will have an empty file in recovery, nearly all the time, but the recovery code already handles that case.
- presently, a tablet doesn't have a marker for a log it did not use. Many more tablets will attempt to recover when it is unnecessary.
This would also address