The client hangs if the execution has two time-outs. After debugging into the issue, we figure out that the resending mechanism has some bugs in it. Here are the detailed description.
Please review the necessity for the invocation of sendCAS(). In our system, we don't have to resend the CAS to process again. Please provide a configuration so that we don't have to resend the CAS every time there is a time out.
Whenever there is the first time-out, BaseUIMAAsynchronousEngine_impl#notifyOnTimout() is invoked and it hangs when it tries to invoke sendCas() at line 2385. I believe the reason is that the sendCAS() is a synchronized method and a potential threading issue causes this thread hang over there. Please be noted that this block is also synchronized.
Then when there is a second time-out, it will be hanging in the very beginning of the method notifyOnTimeout() and cannot clear the time-out CAS from the CAS list, which hangs the entire client.