Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15068

EC2MRS - cassandra-11356 patch seems to break useful broadcast_rpc_address defaulting from 2.1



    • Type: Bug
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Cluster/Gossip
    • Labels:
    • Bug Category:
    • Severity:
    • Complexity:
      Low Hanging Fruit
    • Discovered By:
      User Report


      We have a couple clusters that are using Ec2MultiRegionSnitch that are 2.1.x that we are attempting to upgrade. 

      Our 2.1.x yamls have internal IPs set for broadcast_rpc_address.

      Source of EC2MRS for 2.1: https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java

      The code in 2.1.x EC2MRS seems to not care what you set in the yaml, and overrides to: 

      DatabaseDescriptor.setBroadcastAddress(localPublicAddress); DatabaseDescriptor.setBroadcastRpcAddress(localPublicAddress);

      The code in 2.2.x EC2MRS (https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java) ONLY does this is if the broadcast_rpc_address is somehow null.

      Our rpc_address is set to

      But cassandra will not startup with rpc_address set to and broadcast_rpc_address commetned out, empty, or explicitly null. 

      It would APPEAR in the code that while cassandra normally doesn't like this being blank, for EC2MRS and it's autodetection of private and public IPs using aws metadata urls, this isn't a huge deal. 

      So it would seem that what people have been doing is setting the global ip for broadcast_rpc_address in the yaml, but if the aws instance is hardware rebooted/replaced, the global IP changes and we need to edit the yaml. That sucks, whereas in 2.1 this was not necessary.

      If we blank the rpc_address rather than use, then cqlsh does not work with localhost and that breaks cluster management tooling we have, and it might not be correct.

      I understand this was changed to accomodate CASSANDRA-11356

      It would seem we could reenable beneficial 2.1.x behavior of EC2MRS by either providing a exception for the usual startup of cassandra which complains if broadcast_rpc_address is not set and rpc_address is, or maybe provide a magic value for broadcast_rpc_address to overwrite, or... I don't know.

      Or is there some magic yaml configuration that will allow cassandra to startup but allow the autoset of broadcast_address AND broadcast_rpc_address to the metadata-detected public addresses, rpc_address set to, and the listen_address to the internal VPC address?

      Or am I missing something fundamental here?

      We REALLY don't want to do custom builds. 

      PERHAPS we could do a custom old-style implementation (EC2MRSOld) in a jar and use that?




            • Assignee:
              cowardlydragon Constance Eustace
            • Votes:
              0 Vote for this issue
              2 Start watching this issue


              • Created: