Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-3214

Make CFIF use rpc_endpoint prior to trying endpoint

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Fix Version/s: 0.8.8, 1.0.1
    • Component/s: None
    • Labels:
      None
    • Environment:

      Hadoop 0.20 / Cassandra 0.8.5 / Ubuntu 10.04 /

      Description

      Following up on CASSANDRA-3187 , we probably need to attempt to use the rpc_endpoint address first, and then fall back to the gossip endpoint if we don't get what we want.

        Activity

        Hide
        eldondev Eldon Stegall added a comment -

        Still needs testing, guess I need to spin up an 0.8.6 cluster to do so. Feel free to test if you want to get it in sooner. Feedback welcome!

        Show
        eldondev Eldon Stegall added a comment - Still needs testing, guess I need to spin up an 0.8.6 cluster to do so. Feel free to test if you want to get it in sooner. Feedback welcome!
        Hide
        jbellis Jonathan Ellis added a comment -

        How would fall-back-to-gossip-endpoint be useful? If the two are different, then rpc_address is the only one thrift will be listening on.

        Show
        jbellis Jonathan Ellis added a comment - How would fall-back-to-gossip-endpoint be useful? If the two are different, then rpc_address is the only one thrift will be listening on.
        Hide
        brandon.williams Brandon Williams added a comment -

        It's (possibly) useful in the case that rpc_address is bound to all interfaces, because 0.0.0.0 is not useable.

        Show
        brandon.williams Brandon Williams added a comment - It's (possibly) useful in the case that rpc_address is bound to all interfaces, because 0.0.0.0 is not useable.
        Hide
        jbellis Jonathan Ellis added a comment -

        But it's totally valid to have rpc_ on 0.0.0.0 and listen on something that clients cannot connect to (separate interface, firewalled off, etc). So fundamentally I think it comes down to "you shouldn't use 0.0.0.0 as rpc_address if you're going to use describe_ring."

        Show
        jbellis Jonathan Ellis added a comment - But it's totally valid to have rpc_ on 0.0.0.0 and listen on something that clients cannot connect to (separate interface, firewalled off, etc). So fundamentally I think it comes down to "you shouldn't use 0.0.0.0 as rpc_address if you're going to use describe_ring."
        Hide
        brandon.williams Brandon Williams added a comment -

        That's true, and I agree, but we should still try our hardest and give listen_address a try.

        Show
        brandon.williams Brandon Williams added a comment - That's true, and I agree, but we should still try our hardest and give listen_address a try.
        Hide
        jbellis Jonathan Ellis added a comment -

        +0 from me then

        Show
        jbellis Jonathan Ellis added a comment - +0 from me then
        Hide
        scottfines Scott Fines added a comment -

        The patch file isn't exactly correct.

        In getSubSplits, it still refers to range.endpoints instead of range.rpc_endpoints

        I'm attaching a patch which fixes the typ0; please let me know if it is improperly formatted.

        I tested this patch against a 0.8.7 cluster with a different listen_address than rpc_address, and it appears to work correctly. Not that that is exactly an automated test, but...

        Show
        scottfines Scott Fines added a comment - The patch file isn't exactly correct. In getSubSplits, it still refers to range.endpoints instead of range.rpc_endpoints I'm attaching a patch which fixes the typ0; please let me know if it is improperly formatted. I tested this patch against a 0.8.7 cluster with a different listen_address than rpc_address, and it appears to work correctly. Not that that is exactly an automated test, but...
        Hide
        brandon.williams Brandon Williams added a comment -

        Committed with formatting fixes. Thanks!

        Show
        brandon.williams Brandon Williams added a comment - Committed with formatting fixes. Thanks!
        Hide
        hudson Hudson added a comment -

        Integrated in Cassandra-0.8 #373 (See https://builds.apache.org/job/Cassandra-0.8/373/)
        Make CFIF use rpc_endpoint prior to trying endpoint.
        Patch by Eldon Stegall and Scott Fines, reviewed by brandonwilliams for
        CASSANDRA-3214

        brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183489
        Files :

        • /cassandra/branches/cassandra-0.8/CHANGES.txt
        • /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
        Show
        hudson Hudson added a comment - Integrated in Cassandra-0.8 #373 (See https://builds.apache.org/job/Cassandra-0.8/373/ ) Make CFIF use rpc_endpoint prior to trying endpoint. Patch by Eldon Stegall and Scott Fines, reviewed by brandonwilliams for CASSANDRA-3214 brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183489 Files : /cassandra/branches/cassandra-0.8/CHANGES.txt /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
        Hide
        tjake T Jake Luciani added a comment -

        the accepted patch contains a == bug

        if(endpoint_address == null || endpoint_address == "0.0.0.0")

        Show
        tjake T Jake Luciani added a comment - the accepted patch contains a == bug if(endpoint_address == null || endpoint_address == "0.0.0.0")
        Hide
        jbellis Jonathan Ellis added a comment -

        opened CASSANDRA-3584 for this

        Show
        jbellis Jonathan Ellis added a comment - opened CASSANDRA-3584 for this
        Hide
        derekmcgowan Derek McGowan added a comment -

        There is a second bug in getSubSplits as well, since it is iterating on rpc_endpoints but not checking whether those endpoints are 0.0.0.0 before trying to establish a connection.

        Cassandra.Client client = ConfigHelper.createConnection(host, ConfigHelper.getRpcPort(conf), true);

        Host in this case could be 0.0.0.0, this call is being made before the other check.

        Show
        derekmcgowan Derek McGowan added a comment - There is a second bug in getSubSplits as well, since it is iterating on rpc_endpoints but not checking whether those endpoints are 0.0.0.0 before trying to establish a connection. Cassandra.Client client = ConfigHelper.createConnection(host, ConfigHelper.getRpcPort(conf), true); Host in this case could be 0.0.0.0, this call is being made before the other check.

          People

          • Assignee:
            eldondev Eldon Stegall
            Reporter:
            eldondev Eldon Stegall
            Reviewer:
            Brandon Williams
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development