Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-3569

Failure detector downs should not break streams

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: 2.1.1
    • Labels:
      None

      Description

      CASSANDRA-2433 introduced this behavior just to get repairs to don't sit there waiting forever. In my opinion the correct fix to that problem is to use TCP keep alive. Unfortunately the TCP keep alive period is insanely high by default on a modern Linux, so just doing that is not entirely good either.

      But using the failure detector seems non-sensicle to me. We have a communication method which is the TCP transport, that we know is used for long-running processes that you don't want to incorrectly be killed for no good reason, and we are using a failure detector tuned to detecting when not to send real-time sensitive request to nodes in order to actively kill a working connection.

      So, rather than add complexity with protocol based ping/pongs and such, I propose that we simply just use TCP keep alive for streaming connections and instruct operators of production clusters to tweak net.ipv4.tcp_keepalive_

      {probes,intvl}

      as appropriate (or whatever equivalent on their OS).

      I can submit the patch. Awaiting opinions.

        Attachments

        1. 3569_v1.txt
          14 kB
          Joshua McKenzie
        2. 3569-2.0.txt
          0.6 kB
          Yuki Morishita

          Issue Links

            Activity

              People

              • Assignee:
                JoshuaMcKenzie Joshua McKenzie
                Reporter:
                scode Peter Schuller
                Reviewer:
                Marcus Eriksson
              • Votes:
                1 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: