Uploaded image for project: 'Apache Cassandra'
  1. Apache Cassandra
  2. CASSANDRA-2498

Improve read performance in update-intensive workload

Agile BoardAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Low
    • Resolution: Fixed
    • 1.0.0
    • None

    Description

      Read performance in an update-heavy environment relies heavily on compaction to maintain good throughput. (This is not the case for workloads where rows are only inserted once, because the bloom filter keeps us from having to check sstables unnecessarily.)

      Very early versions of Cassandra attempted to mitigate this by checking sstables in descending generation order (mostly equivalent to descending mtime): once all the requested columns were found, it would not check any older sstables.

      This was incorrect, because data timestamp will not correspond to sstable timestamp, both because compaction has the side effect of "refreshing" data to a newer sstable, and because hintead handoff may send us data older than what we already have.

      Instead, we could create a per-sstable piece of metadata containing the most recent (client-specified) timestamp for any column in the sstable. We could then sort sstables by this timestamp instead, and perform a similar optimization (if the remaining sstable client-timestamps are older than the oldest column found in the desired result set so far, we don't need to look further). Since under almost every workload, client timestamps of data in a given sstable will tend to be similar, we expect this to cut the number of sstables down proportionally to how frequently each column in the row is updated. (If each column is updated with each write, we only have to check a single sstable.)

      This may also be useful information when deciding which SSTables to compact.

      (Note that this optimization is only appropriate for named-column queries, not slice queries, since we don't know what non-overlapping columns may exist in older sstables.)

      Attachments

        1. supersede-name-filter-collations.patch
          14 kB
          Daniel Doubleday
        2. 2498-v4.txt
          25 kB
          Jonathan Ellis
        3. 2498-v3.txt
          22 kB
          Jonathan Ellis
        4. 2498-v2.txt
          15 kB
          Jonathan Ellis

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jbellis Jonathan Ellis Assign to me
            jbellis Jonathan Ellis
            Jonathan Ellis
            Jonathan Ellis
            Votes:
            2 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment