Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-6812

Iterative Memtable->SSTable Replacement

    XMLWordPrintableJSON

Details

    Description

      In an ideal world we wouldn't flush any memtable until we were almost completely out of room. The problem with this approach (and in fact whenever we currently do run out of room) is that flushing an entire memtable is a slow process, and so write latencies spike dramatically during this interval.

      The solution to this is, in principle, quite straight forward: As we write chunks of the new sstable and its index, open them up immediately for reading, and free the memory associated with the portion of the file that has been written so that it can be reused immediately for writing. This way whilst latency will increase for the duration of the flush, the max latency experienced during this time should be no greater than the time taken to flush a few chunks, which should still be on the order of milliseconds, not seconds.

      This depends on CASSANDRA-6689 and CASSANDRA-6694, so that we can reclaim arbitrary portions of the allocated memory prior to a complete flush.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              benedict Benedict Elliott Smith
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated: