Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10111

reconnecting snitch can bypass cluster name check

Agile BoardAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Normal

    Description

      Setup:

      • Two clusters: A & B
      • Both are two DC cluster
      • Both use GossipingPropertyFileSnitch with different listen_address/broadcast_address

      A new node was added to cluster A with a broadcast_address of an existing node in cluster B (due to an out of data DNS entry). Cluster B added all of the nodes from cluster A, somehow bypassing the cluster name mismatch check for this nodes. The first reference to cluster A nodes in cluster B logs is when then were added:

       INFO [GossipStage:1] 2015-08-17 15:08:33,858 Gossiper.java (line 983) Node /8.37.70.168 is now part of the cluster
      

      Cluster B nodes then tried to gossip to cluster A nodes, but cluster A kept them out with 'ClusterName mismatch'. Cluster B however tried to send to send reads/writes to cluster A and general mayhem ensued.

      Obviously this is a Bad (TM) config that Should Not Be Done. However, since the consequence of crazy merged clusters are really bad (the reason there is the name mismatch check in the first place) I think the hole is reasonable to plug. I'm not sure exactly what the code path is that skips the check in GossipDigestSynVerbHandler.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jkni Joel Knighton Assign to me
            cburroughs Chris Burroughs
            Joel Knighton

            Dates

              Created:
              Updated:

              Slack

                Issue deployment