Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-5569

Every stream operation requires checking indexes in every SSTable

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Low
    • Resolution: Fixed
    • Fix Version/s: 1.2.6
    • Component/s: None
    • Labels:
    • Severity:
      Low

      Description

      It looks like there's a streaming performance issue when leveled compaction and vnodes get together. To get the candidate set of chunks to stream, the streaming system gets references to every SSTable for a CF. This is probably a perfectly reasonable assumption for non-vnode cases, because the data being streamed is likely distributed across the full SSTable set. This is also probably a perfectly reasonable assumption for size-tiered compaction, because the data is, again, likely distributed across the full SSTable set. However, for each vnode repair performed on LCS CF's, this scan across potentially tens of thousands of SSTables is wasteful considering that only a small percentage of them will actually have data for a given range.

      This manifested itself as "hanging" repair operations with tasks backing up on the MiscStage thread pool.

      The attached patch changes the streaming code so that for a given range, only SSTables for the requested range are checked to be included in streaming.

        Attachments

        1. 5569.txt
          2 kB
          Rick Branson
        2. 5569-v2.txt
          8 kB
          Rick Branson
        3. 5569-v3.txt
          10 kB
          Rick Branson

          Activity

            People

            • Assignee:
              rbranson Rick Branson
              Reporter:
              rbranson Rick Branson
              Authors:
              Rick Branson
              Reviewers:
              Yuki Morishita
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: