Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Fix Version/s: 4.x
    • Component/s: None
    • Labels:
      None

      Description

      Currently storage_port must be configured identically on all nodes in a cluster and it is assumed that this is the case when connecting to a remote node.

      This prevents running in any environment that requires multiple nodes to be able to bind to the same network interface, such as with many automatic provisioning/deployment frameworks.

      The current solutions seems to be

      • use a separate network interface for each node deployed to the same box. This puts a big requirement on IP allocation at large scale.
      • allow multiple clusters to be provisioned from the same resource pool, but restrict allocation to a maximum of one node per host from each cluster, assuming each cluster is running on a different storage port.

      It would make operations much simpler in these kind of environments if the environment provisioning the resources could assign the ports to be used when bringing up a new node on shared hardware.

      The changes required would be at least the following:
      1. configure seeds as IP:port instead of just IP
      2. gossip the storage port as part of a node's ApplicationState
      3. refer internally to nodes by hostID instead of IP, since there will be multiple nodes with the same IP

      (1) & (2) are mostly trivial and I already have a patch for these. The bulk of the work to enable this is (3), and I would structure this as a separate pre-requisite patch.

        Issue Links

          Activity

          Hide
          samunuru Raju S added a comment -

          so prior versions of 3.X, can we not able to configure different ports for each node in cluster for rpc_port and storage_port?

          Show
          samunuru Raju S added a comment - so prior versions of 3.X, can we not able to configure different ports for each node in cluster for rpc_port and storage_port?
          Hide
          brandon.williams Brandon Williams added a comment -

          No.

          Show
          brandon.williams Brandon Williams added a comment - No.
          Hide
          kohlisankalp sankalp kohli added a comment -
          Show
          kohlisankalp sankalp kohli added a comment - cc Patrick McFadin
          Hide
          aweisberg Ariel Weisberg added a comment - - edited
          Code utests dtests
          trunk utests dtests

          dtest changes Ignore the upgrade_manifest.py I will back those out once I have stabilized the upgrade tests. There are several bugs unrelated to this change that cause the upgrade tests to fail. They also test released versions with bugs which won't start passing until more releases happen.

          Show
          aweisberg Ariel Weisberg added a comment - - edited Code utests dtests trunk utests dtests dtest changes Ignore the upgrade_manifest.py I will back those out once I have stabilized the upgrade tests. There are several bugs unrelated to this change that cause the upgrade tests to fail. They also test released versions with bugs which won't start passing until more releases happen.
          Hide
          aweisberg Ariel Weisberg added a comment -

          This introduces a client library impacting change for CASSANDRA-11596 that is currently disabled by a flag so that client libraries that claim to support the next version of the protocol continue to work.

          Ron Kuris how are things going with the Java client?

          Currently no one has done the Python driver or any of the other drivers. Not sure how we go about integrating changes that require drivers to support a newer version of the protocol.

          Show
          aweisberg Ariel Weisberg added a comment - This introduces a client library impacting change for CASSANDRA-11596 that is currently disabled by a flag so that client libraries that claim to support the next version of the protocol continue to work. Ron Kuris how are things going with the Java client? Currently no one has done the Python driver or any of the other drivers. Not sure how we go about integrating changes that require drivers to support a newer version of the protocol.
          Hide
          kohlisankalp sankalp kohli added a comment -

          Brandon Williams I see that you are assigned the reviewer of this JIRA way back in 2014. Will you be reviewing this or we find someone else?

          Show
          kohlisankalp sankalp kohli added a comment - Brandon Williams I see that you are assigned the reviewer of this JIRA way back in 2014. Will you be reviewing this or we find someone else?
          Hide
          brandon.williams Brandon Williams added a comment -

          Cleared the reviewer.

          Show
          brandon.williams Brandon Williams added a comment - Cleared the reviewer.
          Hide
          rustyrazorblade Jon Haddad added a comment -

          Is this patch destined for 4.0?

          Show
          rustyrazorblade Jon Haddad added a comment - Is this patch destined for 4.0?
          Hide
          aweisberg Ariel Weisberg added a comment -

          I think that depends on the 4.0 release date. This was really kind of sort of ready to go in quite a while ago, but we decided to make it depend on Netty and using a single port for both SSL and non-SSL so that we would only use a single port throughout the code base rather than two and transition away from two ports later.

          Jason says that CASSANDRA-10404 is almost done. I need to do a lot of rebasing. On to of Netty in C*, but also the Java and Python driver, and those haven't been reviewed yet.

          Show
          aweisberg Ariel Weisberg added a comment - I think that depends on the 4.0 release date. This was really kind of sort of ready to go in quite a while ago, but we decided to make it depend on Netty and using a single port for both SSL and non-SSL so that we would only use a single port throughout the code base rather than two and transition away from two ports later. Jason says that CASSANDRA-10404 is almost done. I need to do a lot of rebasing. On to of Netty in C*, but also the Java and Python driver, and those haven't been reviewed yet.

            People

            • Assignee:
              aweisberg Ariel Weisberg
              Reporter:
              soverton Sam Overton
              Reviewer:
              Jason Brown
            • Votes:
              7 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

              • Created:
                Updated:

                Development