Uploaded image for project: 'Apache AsterixDB'
  1. Apache AsterixDB
  2. ASTERIXDB-2168

Flushed LSM components receive non-increasing LSNs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: STO - Storage
    • Labels:
      None

      Description

      Every time when we flush a memory component for a durable dataset, we always force a log record, and use the LSN of the log record as the FLUSH_LSN of the flushed disk component. Since LSNs are strictly increasing, then FLUSH_LSNs of a index on a partition should be strictly increasing as well. However, this properly may not hold in some case.

      One cause (and there may be other causes as well) is that is the ioOperationCallback.afterFinalize is not synchronized on opTracker. When a flush is finished, we set flushRequested to false and advance the readIndex. And in updateLSN, if a component has been requested to flush, we cannot reset the LSN because otherwise we may lose the previous LSN.

      Here is the issue. A memory component has been flushed. Then the next flush is to be scheduled (coming from other partitions), and we try to updateLSN. But this time afterFinalize is not called yet, and we'll skip the newest LSN (which we shouldn't). Then the flush is scheduled, and we call afterFinalize on the previous flushed component (which is already late at this moment).

        Attachments

          Activity

            People

            • Assignee:
              luochen01 Chen Luo
              Reporter:
              luochen01 Chen Luo
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: