Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.8.0
    • Component/s: master, tserver
    • Labels:
      None

      Description

      After reading the proposal on HBASE-10278, I realized there are many ways to make the Accumulo WAL roll-over faster.

      1. Open two WALogs, but use only one until it reaches the WALog roll-over size
      2. Rollover consists only of swapping the writers
      3. WALog roll consists of the final close, which can happen in parallel
      4. Don't mark the tablets with log entries: they are already marked with the tserver
      5. The tserver can make notes about the logs-in-use in the metadata table(s) as part of opening the log.
      6. The master can copy the log entries to tablets while unassigning them, piggybacking on the unassigment mutation.
      7. Tablet servers can remove their current log entries from the metadata tables when they have no tablets using them.

      There are two issues:

      1. tablets will have an empty file in recovery, nearly all the time, but the recovery code already handles that case.
      2. presently, a tablet doesn't have a marker for a log it did not use. Many more tablets will attempt to recover when it is unnecessary.

      This would also address ACCUMULO-2889.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ecn Eric Newton
                Reporter:
                ecn Eric Newton
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 50m
                  3h 50m