Cassandra
  1. Cassandra
  2. CASSANDRA-3452

Create an 'infinite bootstrap' mode for sampling live traffic

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 1.1.0
    • Component/s: None
    • Labels:
      None

      Description

      You may want to, for example, test a new compaction strategy with live traffic to see how it will fare. In this mode, the node would follow the bootstrap procedure as normal, but never fully join the ring.

        Activity

        Hide
        Hudson added a comment -

        Integrated in Cassandra #1266 (See https://builds.apache.org/job/Cassandra/1266/)
        Add 'write survey' mode that bootstraps but does not join.
        Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-3452

        brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1222425
        Files :

        • /cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
        Show
        Hudson added a comment - Integrated in Cassandra #1266 (See https://builds.apache.org/job/Cassandra/1266/ ) Add 'write survey' mode that bootstraps but does not join. Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-3452 brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1222425 Files : /cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
        Hide
        Brandon Williams added a comment -

        Committed 0001 to 1.0.7 and 0002 to 1.1.

        Show
        Brandon Williams added a comment - Committed 0001 to 1.0.7 and 0002 to 1.1.
        Hide
        Jonathan Ellis added a comment -

        +1

        (I'd be okay with 01 in 1.0.7, but 02 should be 1.1 IMO)

        Show
        Jonathan Ellis added a comment - +1 (I'd be okay with 01 in 1.0.7, but 02 should be 1.1 IMO)
        Hide
        Brandon Williams added a comment -

        Patch to create a "write survey" mode enabled via -Dcassandra.write_survey=true, where the node will bootstrap as usual but NOT join the ring unless SS.joinRing is called later (as you would with cassandra.join_ring=false. Also a patch jmx-enable the setting of the compaction strategy on a CF so these are more useful as a whole.

        Show
        Brandon Williams added a comment - Patch to create a "write survey" mode enabled via -Dcassandra.write_survey=true, where the node will bootstrap as usual but NOT join the ring unless SS.joinRing is called later (as you would with cassandra.join_ring=false. Also a patch jmx-enable the setting of the compaction strategy on a CF so these are more useful as a whole.
        Hide
        Brandon Williams added a comment -

        IE. You could convert a single node to a compressed LCS node. It would take reads and writes, but you would not do any repairs, or joins, or moves or possibly any other schema changes. At some point you would evaluate the performance vs another node in your cluster, look at your performance and make the decision. If you want the change to happen across the cluster or be undone.

        If you want to localize changes to a node then JMX is the way to do it. If the node is going to take reads, it might as well be part of the cluster.

        Show
        Brandon Williams added a comment - IE. You could convert a single node to a compressed LCS node. It would take reads and writes, but you would not do any repairs, or joins, or moves or possibly any other schema changes. At some point you would evaluate the performance vs another node in your cluster, look at your performance and make the decision. If you want the change to happen across the cluster or be undone. If you want to localize changes to a node then JMX is the way to do it. If the node is going to take reads, it might as well be part of the cluster.
        Hide
        Edward Capriolo added a comment -

        This is pretty interesting. Ideally I was thinking of being able to do a head to head test.

        IE. You could convert a single node to a compressed LCS node. It would take reads and writes, but you would not do any repairs, or joins, or moves or possibly any other schema changes. At some point you would evaluate the performance vs another node in your cluster, look at your performance and make the decision. If you want the change to happen across the cluster or be undone.

        I know my request is asking something to the effect of 'Help me test in production', But coming up with hardware and a system to do an actual test of production load is difficult. If we were able to restrict a schema update to one node we have a way to mitigate risk. For a major change like compression or LCS that will drastically reshape a lot of physical data on disk and could drastically change performance hopefully in a good way, but if it did not work out well, performance could suffer for a long time while the underlying data tables rewrite themselves and you get back to where you started.

        Show
        Edward Capriolo added a comment - This is pretty interesting. Ideally I was thinking of being able to do a head to head test. IE. You could convert a single node to a compressed LCS node. It would take reads and writes, but you would not do any repairs, or joins, or moves or possibly any other schema changes. At some point you would evaluate the performance vs another node in your cluster, look at your performance and make the decision. If you want the change to happen across the cluster or be undone. I know my request is asking something to the effect of 'Help me test in production', But coming up with hardware and a system to do an actual test of production load is difficult. If we were able to restrict a schema update to one node we have a way to mitigate risk. For a major change like compression or LCS that will drastically reshape a lot of physical data on disk and could drastically change performance hopefully in a good way, but if it did not work out well, performance could suffer for a long time while the underlying data tables rewrite themselves and you get back to where you started.

          People

          • Assignee:
            Brandon Williams
            Reporter:
            Brandon Williams
            Reviewer:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development