UIMA
  1. UIMA
  2. UIMA-2065

UIMA AS client needs ShutdownHook to close broker connection

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.1AS
    • Component/s: Async Scaleout
    • Labels:
      None

      Description

      If an application exits without calling stop method on the UIMA AS client, an ugly exception is thrown by the broker:
      java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:168)
      at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBuff
      eredInputStream.java:50)
      at org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.ja
      va:575)
      at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBuff
      eredInputStream.java:58)
      at org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.ja
      va:560)
      at java.io.DataInputStream.readInt(DataInputStream.java:370)
      at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.
      java:269)
      at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTranspo
      rt.java:226)
      at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.jav
      a:218)
      at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:
      201)
      at java.lang.Thread.run(Thread.java:619)

      This exception is thrown by the broker when it realizes that the socket connection to a client closed unexpectedly. Add a Shutdown Hook to the client to enable cleanup even though the application exits without calling stop().

        Activity

        Hide
        Jerry Cwiklik added a comment -

        Added ShutdownHook to enable clean shutdown of UIMA AS client. As part of this, a new thread was introduced in the client to process reply messages. Previously reply messages were processed in the AMQ dispatch thread, but I was getting hangs when ShutdownHook was activated. After painful debugging it looked like there was a deadlock between dispatch thread and the ShutdownHook thread. Reply message is delivered to the UIMA AS client onMessage() method from within critical region in AMQ dispatch code. The dispatch thread is used to process the reply and to notify application callback listener. In the listener an application calls System.exit() activating the ShutdownHook thread. The jvm does not release the dispatch thread however and pins it to the ShutdownHook thread via join(). The dispatch thread waits for the ShutdownHook to finish which never happens since the ShutdownHook invokes AMQ close() method on a connection which attempts to acquire the same lock used by the dispatch thread. Deadlock. The solution is to create a dedicated thread in UIMA AS client onMessage() method to process replies and return the dispatch thread back to its pool.

        Show
        Jerry Cwiklik added a comment - Added ShutdownHook to enable clean shutdown of UIMA AS client. As part of this, a new thread was introduced in the client to process reply messages. Previously reply messages were processed in the AMQ dispatch thread, but I was getting hangs when ShutdownHook was activated. After painful debugging it looked like there was a deadlock between dispatch thread and the ShutdownHook thread. Reply message is delivered to the UIMA AS client onMessage() method from within critical region in AMQ dispatch code. The dispatch thread is used to process the reply and to notify application callback listener. In the listener an application calls System.exit() activating the ShutdownHook thread. The jvm does not release the dispatch thread however and pins it to the ShutdownHook thread via join(). The dispatch thread waits for the ShutdownHook to finish which never happens since the ShutdownHook invokes AMQ close() method on a connection which attempts to acquire the same lock used by the dispatch thread. Deadlock. The solution is to create a dedicated thread in UIMA AS client onMessage() method to process replies and return the dispatch thread back to its pool.
        Hide
        Jerry Cwiklik added a comment -

        Forgot to check in BaseUimAsynchronousEngineCommon_impl

        Show
        Jerry Cwiklik added a comment - Forgot to check in BaseUimAsynchronousEngineCommon_impl
        Hide
        Jerry Cwiklik added a comment -

        Checked in changes described previously.

        Show
        Jerry Cwiklik added a comment - Checked in changes described previously.

          People

          • Assignee:
            Jerry Cwiklik
            Reporter:
            Jerry Cwiklik
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development