Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-7460

Send source sstable level when bootstrapping or replacing nodes

    Details

      Description

      When replacing or bootstrapping a new node we can keep the source sstable level to avoid doing alot of compaction after bootstrap

        Issue Links

          Activity

          Hide
          krummas Marcus Eriksson added a comment -

          attaching 2.1-rebased patch for this

          • send whether we can keep sstable levels in StreamInitMessage
          • send sstable level in FileMessageHeader
          • during bootstrap, only do STCS in L0 to avoid getting overlap in L1

          Note that this still does not handle backwards compatibility as it is quite complicated to fix and should probably be done in another ticket.

          My suggested fix is that when the stream initiator sends a StreamInitMessage with CURRENT_VERSION, the receiver needs to reply with the max version that both nodes supports and the messages are then sent using that version.

          Another approach would be to figure out what version the remote node is before deciding on streaming version, but that would mean we have to bump messaging version whenever we change something in a stream message which we probably shouldn't.

          WDYT Yuki Morishita ?

          Show
          krummas Marcus Eriksson added a comment - attaching 2.1-rebased patch for this send whether we can keep sstable levels in StreamInitMessage send sstable level in FileMessageHeader during bootstrap, only do STCS in L0 to avoid getting overlap in L1 Note that this still does not handle backwards compatibility as it is quite complicated to fix and should probably be done in another ticket. My suggested fix is that when the stream initiator sends a StreamInitMessage with CURRENT_VERSION, the receiver needs to reply with the max version that both nodes supports and the messages are then sent using that version. Another approach would be to figure out what version the remote node is before deciding on streaming version, but that would mean we have to bump messaging version whenever we change something in a stream message which we probably shouldn't. WDYT Yuki Morishita ?
          Hide
          jbellis Jonathan Ellis added a comment -

          If we're serious about doing shorter releases, this is the kind of thing that should go into 3.0 and not 2.1.

          Show
          jbellis Jonathan Ellis added a comment - If we're serious about doing shorter releases, this is the kind of thing that should go into 3.0 and not 2.1.
          Hide
          krummas Marcus Eriksson added a comment -

          sure

          Show
          krummas Marcus Eriksson added a comment - sure
          Hide
          krummas Marcus Eriksson added a comment -

          trunk-rebased patch attached

          Show
          krummas Marcus Eriksson added a comment - trunk-rebased patch attached
          Hide
          jbellis Jonathan Ellis added a comment -

          switching reviewer to Aleksey Yeschenko

          Show
          jbellis Jonathan Ellis added a comment - switching reviewer to Aleksey Yeschenko
          Hide
          kohlisankalp sankalp kohli added a comment -

          Marcus Eriksson The patch is not applying to trunk. Can you please update new one.

          Show
          kohlisankalp sankalp kohli added a comment - Marcus Eriksson The patch is not applying to trunk. Can you please update new one.
          Hide
          krummas Marcus Eriksson added a comment -
          Show
          krummas Marcus Eriksson added a comment - rebased and pushed here: https://github.com/krummas/cassandra/commits/marcuse/7460
          Hide
          iamaleksey Aleksey Yeschenko added a comment -

          LGTM, +1

          Also, agree with the backwards compatibility points. It belongs to a separate ticket, but it must be done - users with huge clusters should be able to upgrade seamlessly without noticing that the upgrade is in process at all (need this and CASSANDRA-6038 to make it happen).

          Show
          iamaleksey Aleksey Yeschenko added a comment - LGTM, +1 Also, agree with the backwards compatibility points. It belongs to a separate ticket, but it must be done - users with huge clusters should be able to upgrade seamlessly without noticing that the upgrade is in process at all (need this and CASSANDRA-6038 to make it happen).
          Hide
          krummas Marcus Eriksson added a comment -

          thanks, committed, created CASSANDRA-8110 for backwards compatible streaming

          Show
          krummas Marcus Eriksson added a comment - thanks, committed, created CASSANDRA-8110 for backwards compatible streaming

            People

            • Assignee:
              krummas Marcus Eriksson
              Reporter:
              krummas Marcus Eriksson
              Reviewer:
              Aleksey Yeschenko
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development