Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-3606

Node sometimes fails to detect broken connection

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3
    • Component/s: general
    • Labels:
      None

      Description

      Here is test reproducing issue https://github.com/rossdanderson/IgniteDeadlock.

      When I run this test observe this sequence:

      • server starts
      • client starts
      • server sends 2000 messages to client, on client node communication backpressure pauses reads
      • server gets write timeout and closes socket
      • for some reason client does not detect that existing connection was broken and thinks that connection is still established (most probably because reads are paused and node does not try to access connection)
      • when server tries to re-connec, client sees that connection already established and rejects connection, so server constantly tries to reconnect and does not exit from reconnect loop:
        "main" prio=6 tid=0x0000000001f4a000 nid=0x3588 waiting on condition [0x00000000021ed000]
           java.lang.Thread.State: TIMED_WAITING (sleeping)
        	at java.lang.Thread.sleep(Native Method)
        	at org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7414)
        	at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055)
        	at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1970)
        	at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1936)
        	at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1304)
        	at org.apache.ignite.internal.managers.communication.GridIoManager.sendOrderedMessage(GridIoManager.java:1540)
        

        Attachments

        1. logs.zip
          9.76 MB
          Ksenia Rybakova

          Activity

            People

            • Assignee:
              krybakova Ksenia Rybakova
              Reporter:
              sboikov Semen Boikov
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: