Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 1.0.0
    • Component/s: Core
    • Labels:
      None

      Description

      Streaming currently is a two-pass operation: one to write the Data component do disk from the socket, then another to build the index and bloom filter from it. This means we do about 2x the i/o we would if we created the index and BF during the original write.

      For node movement this was not considered to be a Big Deal because the stream target is not a member of the ring, so we can be inefficient without hurting live queries. But optimizing node movement to not require un/rebootstrap (CASSANDRA-1427) and bulk load (CASSANDRA-1278) mean we can stream to live nodes too.

      The main obstacle here is we don't know how many keys will be in the new sstable ahead of time, which we need to size the bloom filter correctly. We can solve this by including that information (or a close approximation) in the stream setup – the source node can calculate that without hitting disk from the in-memory index summary.

        Activity

        Gavin made changes -
        Workflow patch-available, re-open possible [ 12752841 ] reopen-resolved, no closed status, patch-avail, testing [ 12755584 ]
        Gavin made changes -
        Workflow no-reopen-closed, patch-avail [ 12614207 ] patch-available, re-open possible [ 12752841 ]
        Jonathan Ellis made changes -
        Resolution Fixed [ 1 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Yuki Morishita [ yukim ]
        Reviewer jbellis
        Yuki Morishita made changes -
        Attachment trunk-2677.txt [ 12486319 ]
        Jonathan Ellis made changes -
        Fix Version/s 1.0 [ 12316349 ]
        Fix Version/s 0.8.2 [ 12316645 ]
        Jonathan Ellis made changes -
        Assignee Sylvain Lebresne [ slebresne ]
        Jonathan Ellis made changes -
        Field Original Value New Value
        Fix Version/s 0.8.2 [ 12316645 ]
        Fix Version/s 0.8.1 [ 12316368 ]
        Jonathan Ellis created issue -

          People

          • Assignee:
            Yuki Morishita
            Reporter:
            Jonathan Ellis
            Reviewer:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development