Uploaded image for project: 'CXF'
  1. CXF
  2. CXF-7109

ClientCallback may be invoked twice when Async HTTP Transport is used



    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.1.8
    • 3.1.9, 3.2.0
    • Core, Transports
    • None
    • RHEL 6.8

    • Unknown


      With Async HTTP Transport (cxf-rt-transports-http-hc) enabled, the ClientCallback object passed for the invocation of the client may be called twice when a failure occurs during the invocation:

      Client client = ...
      client.invoke(new ClientCallback { ... }, opName, params);

      At this moment, this issue is reproduced only on RHEL 6.8, but code analysis shows that it's a general multithreading issue with Async HTTP Transport and in theory it can happen on any platform.

      Here is what I've found with the code analysis:

      If for some reason the target endpoint is not reachable, this issue can happen when the callback for Apache HC HttpAsyncClient [1] is invoked and thus shuts down SharedOutputBuffer earlier than HTTPConduit tries to write it before closing the conduit itself [2].
      [1] https://github.com/apache/cxf/blob/cxf-3.1.8/rt/transports/http-hc/src/main/java/org/apache/cxf/transport/http/asyncclient/AsyncHTTPConduit.java#L464
      [2] https://github.com/apache/cxf/blob/cxf-3.1.8/rt/transports/http-hc/src/main/java/org/apache/cxf/transport/http/asyncclient/SharedOutputBuffer.java#L231

      When the above issue happens, the interceptor chain's fault observer is eventually invoked twice, once from the response workqueue thread [3] that is spawn by the Apache HC HttpAsyncClient callback and the second time from the requesting thread at the interceptor chain [4]; Thus the client callback is also invoked twice [5]!
      [3] https://github.com/apache/cxf/blob/cxf-3.1.8/rt/transports/http/src/main/java/org/apache/cxf/transport/http/HTTPConduit.java#L1177
      [4] https://github.com/apache/cxf/blob/cxf-3.1.8/core/src/main/java/org/apache/cxf/phase/PhaseInterceptorChain.java#L366
      [5] https://github.com/apache/cxf/blob/cxf-3.1.8/core/src/main/java/org/apache/cxf/interceptor/ClientOutFaultObserver.java#L59

      Why it happens so reliably only on RHEL 6.8 is still unclear (probably due to some thread scheduling logic specific to the kernel for RHEL 6.8), but since it's essentially an ordering issue for two distinct threads, in theory it can happen on any platforms. In general, CXF should really enforce that every client callback be invoked once and only once.


        Issue Links



              ffang Freeman Yue Fang
              tadayosi Tadayoshi Sato
              0 Vote for this issue
              3 Start watching this issue