[CXF-5574] CXF JAX-RS loops infinitely when server closes connection on receipt of message

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 2.7.8, 2.7.10
• Component/s: JAX-RS

Environment

Ubuntu 12.10 64bit

• Estimated Complexity: Unknown
• Assignee: Sergey Beryozkin
• Reporter: jd
• Created: 21/Feb/14 08:35
• Updated: 11/Jul/14 12:54
• Resolved: 21/Feb/14 16:30

Description

I am using CXF Jax-RS using WebClient.get(InvocationCallback<T> callback). Once server receives the request, the server closes connection (without sending any response). This causes CXF Jax-RS to go in an infinite loop.

Using eclipse, I was able to put breakpoints at locations described below and was able to obtain stack traces that proves the problem. Since more than one threads become involved, stack is broken into multiple parts.

Please help resolve this issue as this prevents using CXF JAX-RS in production environment.

The stack traces are as follows:

Stack 1:
-------
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream(HTTPConduit$WrappedOutputStream).handleResponseOnWorkqueue(boolean, boolean) line: 1126
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.handleResponseAsync() line: 227
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream(HTTPConduit$WrappedOutputStream).handleResponse() line: 1506
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream(HTTPConduit$WrappedOutputStream).close() line: 1310
URLConnectionHTTPConduit(AbstractConduit).close(Message) line: 56
URLConnectionHTTPConduit(HTTPConduit).close(Message) line: 628
MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(Message) line: 62
PhaseInterceptorChain.doIntercept(Message) line: 272
WebClient(AbstractClient).doRunInterceptorChain(Message) line: 634
WebClient.doInvokeAsync(String, Object, Class<?>, Type, Class<?>, Type, InvocationCallback<T>) line: 942
WebClient.doInvokeAsyncCallback(String, Object, Class<?>, Type, InvocationCallback<T>) line: 914
WebClient.post(Object, InvocationCallback<T>) line: 450
MyApplicationHandler.sendRequest(MyApplicationHandler$CPendingRequest) line: 298

Stack 2:
-------
Runnable on line 1126 of HttpConduit is run by an executor thread.
handleResponseInternal on 1129 of HttpConduit throws java.net.SocketException:
Unexpected end of file from server causing 1131 catch block to run.

The interceptor chain includes
Chain org.apache.cxf.phase.PhaseInterceptorChain@701fe362. Current flow:
receive [PolicyInInterceptor]
pre-protocol-frontend [ClientResponseFilterInterceptor]
unmarshal [ClientAsyncResponseInterceptor]

Henceforth the stack is as follows:

URLConnectionHTTPConduit$URLConnectionWrappedOutputStream(HTTPConduit$WrappedOutputStream).handleResponseOnWorkqueue(boolean, boolean) line: 1126
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.handleResponseAsync() line: 227
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream(HTTPConduit$WrappedOutputStream).handleResponse() line: 1506
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream(HTTPConduit$WrappedOutputStream).close() line: 1310
URLConnectionHTTPConduit(AbstractConduit).close(Message) line: 56
URLConnectionHTTPConduit(HTTPConduit).close(Message) line: 628
UpfrontConduitSelector(AbstractConduitSelector).complete(Exchange) line: 185
WebClient(AbstractClient).preProcessResult(Message) line: 556
WebClient.handleAsyncResponse(Message) line: 979
WebClient.access$100(WebClient, Message) line: 80
WebClient$ClientAsyncResponseInterceptor.handleMessage(Message) line: 1265
PhaseInterceptorChain.doIntercept(Message) line: 272
ClientMessageObserver.onMessage(Message) line: 56
HTTPConduit$WrappedOutputStream$1.run() line: 1138
AutomaticWorkQueueImpl$3.run() line: 428
AutomaticWorkQueueImpl$1(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1110
ThreadPoolExecutor$Worker.run() line: 603
AutomaticWorkQueueImpl$AWQThreadFactory$1.run() line: 353
Thread.run() line: 722

Stack 3:
-------
MyApplicationHandler$CPendingRequest.failed(ClientException) line: 657
JaxrsClientCallback<T>.handleException(Map<String,Object>, Throwable) line: 90
WebClient.handleAsyncResponse(Message) line: 988
WebClient.access$100(WebClient, Message) line: 80
WebClient$ClientAsyncResponseInterceptor.handleMessage(Message) line: 1265
PhaseInterceptorChain.doIntercept(Message) line: 272
ClientMessageObserver.onMessage(Message) line: 56
HTTPConduit$WrappedOutputStream$1.run() line: 1138
AutomaticWorkQueueImpl$3.run() line: 428
AutomaticWorkQueueImpl$1(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1110
ThreadPoolExecutor$Worker.run() line: 603
AutomaticWorkQueueImpl$AWQThreadFactory$1.run() line: 353
Thread.run() line: 722

Following stacks 4 & 5 now repeat in an infinite loop:

Stack 4:
-------
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream(HTTPConduit$WrappedOutputStream).handleResponseOnWorkqueue(boolean, boolean) line: 1126
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.handleResponseAsync() line: 227
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream(HTTPConduit$WrappedOutputStream).handleResponse() line: 1506
URLConnectionHTTPConduit$URLConnectionWrappedOutputStream(HTTPConduit$WrappedOutputStream).close() line: 1310
URLConnectionHTTPConduit(AbstractConduit).close(Message) line: 56
URLConnectionHTTPConduit(HTTPConduit).close(Message) line: 628
UpfrontConduitSelector(AbstractConduitSelector).complete(Exchange) line: 185
WebClient(AbstractClient).preProcessResult(Message) line: 556
WebClient.handleAsyncResponse(Message) line: 979
WebClient.access$100(WebClient, Message) line: 80
WebClient$ClientAsyncResponseInterceptor.handleMessage(Message) line: 1265
PhaseInterceptorChain.doIntercept(Message) line: 272
ClientMessageObserver.onMessage(Message) line: 56
HTTPConduit$WrappedOutputStream$1.run() line: 1138
AutomaticWorkQueueImpl$3.run() line: 428
AutomaticWorkQueueImpl$1(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1110
ThreadPoolExecutor$Worker.run() line: 603
AutomaticWorkQueueImpl$AWQThreadFactory$1.run() line: 353
Thread.run() line: 722

Stack 5:
-------
MyApplicationHandler$CPendingRequest.failed(ClientException) line: 657
JaxrsClientCallback<T>.handleException(Map<String,Object>, Throwable) line: 90
WebClient.handleAsyncResponse(Message) line: 988
WebClient.access$100(WebClient, Message) line: 80
WebClient$ClientAsyncResponseInterceptor.handleMessage(Message) line: 1265
PhaseInterceptorChain.doIntercept(Message) line: 272
ClientMessageObserver.onMessage(Message) line: 56
HTTPConduit$WrappedOutputStream$1.run() line: 1138
AutomaticWorkQueueImpl$3.run() line: 428
AutomaticWorkQueueImpl$1(ThreadPoolExecutor).runWorker(ThreadPoolExecutor$Worker) line: 1110
ThreadPoolExecutor$Worker.run() line: 603
AutomaticWorkQueueImpl$AWQThreadFactory$1.run() line: 353
Thread.run() line: 722

Breakpoints:
-----------
MyApplicationHandler [line: 657] - failed
MyApplicationHandler [line 298] - sendRequest
AbstractConduit [line: 54] - close(Message)
ClientMessageObserver [line: 56] - ClientMessageObserver
HTTPConduit$WrappedOutputStream$1 [line: 1126] - handleResponseOnWorkqueue(boolean, boolean)
HTTPConduit$WrappedOutputStream [line: 1310] - close()
HTTPConduit$WrappedOutputStream$1 [line: 1129] - handleResponseOnWorkqueue(boolean, boolean)
HTTPConduit$WrappedOutputStream$1 [line: 1131] - handleResponseOnWorkqueue(boolean, boolean)

Issue Links

• duplicates: CXF-5559 Async jaxrs call : Inifinite loop when connexion socket timeout /connexion refused occurs and no action can stop this loop