[ACCUMULO-3423] speed up WAL roll-overs

• Type: Improvement
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.8.0
• Component/s: master, tserver
• Assignee: Eric C. Newton
• Reporter: Eric C. Newton
• Created: 15/Dec/14 22:38
• Updated: 25/Jun/15 14:41
• Resolved: 21/May/15 17:19

Description

After reading the proposal on HBASE-10278, I realized there are many ways to make the Accumulo WAL roll-over faster.

1. Open two WALogs, but use only one until it reaches the WALog roll-over size
2. Rollover consists only of swapping the writers
3. WALog roll consists of the final close, which can happen in parallel
4. Don't mark the tablets with log entries: they are already marked with the tserver
5. The tserver can make notes about the logs-in-use in the metadata table(s) as part of opening the log.
6. The master can copy the log entries to tablets while unassigning them, piggybacking on the unassigment mutation.
7. Tablet servers can remove their current log entries from the metadata tables when they have no tablets using them.

There are two issues:

1. tablets will have an empty file in recovery, nearly all the time, but the recovery code already handles that case.
2. presently, a tablet doesn't have a marker for a log it did not use. Many more tablets will attempt to recover when it is unnecessary.

This would also address ACCUMULO-2889.

Issue Links

• breaks: ACCUMULO-3774 Deadlock after recovering root tablet
• breaks: ACCUMULO-3772 WAL prematurely deleted for root table
• breaks: ACCUMULO-3775 Root tablet had 6,974 walogs
• breaks: ACCUMULO-3787 Multiple locations for Metadata tablet
• is related to: ACCUMULO-2889 Batch metadata table updates for new walogs
• relates to: ACCUMULO-3911 CloseWriteAheadLogReferencesTest consistently failing
• links to: ReviewBoard

Sub-Tasks

• error writing mutations to log: Resolved
• Deadlock after recovering root tablet: Resolved