Accumulo
  1. Accumulo
  2. ACCUMULO-3010

minor compaction after recovery should not merge

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.5.0, 1.5.1, 1.6.0
    • Fix Version/s: 1.6.1, 1.7.0
    • Component/s: tserver
    • Labels:
      None

      Description

      Data ingested during recovery is saved to a file with a minor compaction. By allowing a merging minor compaction, additional failures are possible.

        Activity

        Hide
        Eric Newton added a comment -

        Keith Turner noticed that there is only one reference to the RECOVERY compaction reason.... good catch!

        Show
        Eric Newton added a comment - Keith Turner noticed that there is only one reference to the RECOVERY compaction reason.... good catch!
        Hide
        Josh Elser added a comment - - edited

        Ah, I see, thanks for the explanation Brian Loss.

        In other words, it's proposed to remove the possibility for merging MinC's during the recovery process because the merging MinC itself introduces a higher risk of error. I first read it as a recovery using a merging MinC has explicit errors possible (not just additional risk) which confused me. This makes sense to me and would also be a good idea because a merging MinC would probably slow down recovery time due to the n log n cost of the merging minor recovery over a normal MinC.

        Show
        Josh Elser added a comment - - edited Ah, I see, thanks for the explanation Brian Loss . In other words, it's proposed to remove the possibility for merging MinC's during the recovery process because the merging MinC itself introduces a higher risk of error. I first read it as a recovery using a merging MinC has explicit errors possible (not just additional risk) which confused me. This makes sense to me and would also be a good idea because a merging MinC would probably slow down recovery time due to the n log n cost of the merging minor recovery over a normal MinC.
        Hide
        Brian Loss added a comment -

        Say something like ACCUMULO-3011 happens, and you end up with file references that don't exist. Then during recovery, Accumulo decides to do a merging minor compaction and chooses to merge with a file that doesn't exist. The minor compaction will fail, and it will keep retrying forever and not make progress.

        Show
        Brian Loss added a comment - Say something like ACCUMULO-3011 happens, and you end up with file references that don't exist. Then during recovery, Accumulo decides to do a merging minor compaction and chooses to merge with a file that doesn't exist. The minor compaction will fail, and it will keep retrying forever and not make progress.
        Hide
        Josh Elser added a comment -

        By allowing a merging minor compaction, additional failures are possible.

        Could you please elaborate on a failure condition? It would be much appreciated.

        Show
        Josh Elser added a comment - By allowing a merging minor compaction, additional failures are possible. Could you please elaborate on a failure condition? It would be much appreciated.

          People

          • Assignee:
            Eric Newton
            Reporter:
            Eric Newton
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 1h 10m
              1h 10m

                Development