Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-12905

Retry acquire MV lock on failure instead of throwing WTE on streaming

    XMLWordPrintableJSON

Details

    • Critical

    Description

      Hello,

      I performed two upgrades to the current cluster (currently 15 nodes, 1 DC, private VLAN),
      first it was 2.2.5.1 and repair worked flawlessly,
      second upgrade was to 3.0.9 (with upgradesstables) and also repair worked well,
      then i upgraded 2 weeks ago to 3.9 - and the repair problems started.

      there are several errors types from the system.log (different nodes) :

      • Sync failed between /xxx.xxx.xxx.xxx and /xxx.xxx.xxx.xxx
      • Streaming error occurred on session with peer xxx.xxx.xxx.xxx Operation timed out - received only 0 responses
      • Remote peer xxx.xxx.xxx.xxx failed stream session
      • Session completed with the following error
        org.apache.cassandra.streaming.StreamException: Stream failed

      i use 3.9 default configuration with the cluster settings adjustments (3 seeds, GossipingPropertyFileSnitch).
      streaming_socket_timeout_in_ms is the default (86400000).

      i'm afraid from consistency problems while i'm not performing repair.

      Any ideas?

      Thanks,
      Nir.

      Attachments

        Issue Links

          Activity

            People

              brstgt Benjamin Roth
              zilkanir Nir Zilka
              Benjamin Roth
              Paulo Motta (Deprecated)
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: