Uploaded image for project: 'HttpComponents HttpCore'
  1. HttpComponents HttpCore
  2. HTTPCORE-528

Infinite loop on server disconnect

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.4.9
    • Fix Version/s: 4.4.10, 5.0-beta3
    • Component/s: HttpCore NIO
    • Labels:
      None
    • Environment:
      bare metal Ubuntu 16.04 with kernel 4.4; openjdk1.8.0_172_x64 and Oracle jdk1.8.0_162_x64
      CentOs Docker container with 3.10; openjdk1.8.0_172_x64
      macOS 10.13.5; openjdk1.8.0_172_x64

      Description

      I am seeing HTTP NIO client get into an infinite loop after the server disconnections the connection. See the log output attached; note some content removed for privacy.

       

      Note that the HttpAsyncClient has returned the response and the application has completely processed it. The client maintains the connection, as expected, since the response included `Keep-Alive: timeout=5`. Five seconds after everything is complete, the server closes the TCP connection. The client reacts accordingly: the selector wakes up, does a read of -1 bytes, closes the session and sets a 1 second timeout to close the connection in.

       

      The infinite loop occurs because the selector.select() call constantly returns in AbstractIOReactor.execute()

      readyCount == 1, so events are processed

      processEvent() notes the key is readable and calls:

        session.resetLastRead()

        readable(key);

      Because resetLastRead() is constantly updated, the 1 second timeout is never reached and AbstractIOReactor.timeoutCheck() can never call sessionTimedOut() or close the connection.

       

      Note the entire time this is happening, netstat shows the connection is in CLOSE_WAIT state. The infinite loop continues until the OS keepalive timeout is reached and the connection is cleaned by the OS.

      I am not sure if this epoll / selector behavior is expected or not. However, I have replicated this issue in multiple environments. It seems like the async client should handle this by detecting the condition and closing the connection.

       

      Other notes from this infinite loop state:

      SSLIOSession.updateEventMask() never closes the session either since the state remains `CLOSING`

      AbstractIODispatch.inputReady() does not  read any data from the connection since ssliosession.isAppInputReady() evaluates to false.

      SelectionKeyImpl.interestOps remains 1 (`OP_READ`)

        Attachments

        1. httpdebug.log
          17 kB
          Hunter Presnall

          Activity

            People

            • Assignee:
              olegk Oleg Kalnichevski
              Reporter:
              hpresnall Hunter Presnall
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: