Details
-
Bug
-
Status: Resolved
-
Urgent
-
Resolution: Fixed
-
None
-
Critical
Description
SASI maintains its own in-memory structures for indexing the contents of a base Memtable. On flush, these are simply discarded & replaced with an new instance, whilst the on disk index is built as the base memtable is flushed to SSTables.
SASIIndex implements INotificationHandler and this switching of the index memtable is triggered by receipt of a MemtableRenewedNotification. In the original SASI implementation, one of the necessary modifications to C* was to emit this notification from DataTracker::switchMemtable, but this was overlooked when porting to 3.0. The net result is that the index memtable is never switched out, which eventually leads to OOME.
Simply applying the original modification isn't entirely appropriate though, as it creates a window where it's possible for the index memtable to have been switched, but the flushwriter is yet to finish writing the new index sstables. During this window, index entries will be missing and query results inaccurate.
I propose leaving Tracker::switchMemtable as is, so that INotificationConsumers are only notified from there when truncating, but adding similar notifications in Tracker::replaceFlushed, to fire after the View is updated.
I'm leaning toward re-using MemtableRenewedNotification for this as semantically I don't believe there's any meaningful difference between the flush and truncation cases here. If anyone has a compelling argument for a new notification type though to distinguish the two events, I'm open to hear it.