Uploaded image for project: 'CXF'
  1. CXF
  2. CXF-8951

Concurrent WebClient usage causes massive thread overhead

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 3.6.2, 4.0.3, 3.6.3, 4.0.4
    • 4.1.0, 4.0.5, 3.6.4
    • Core, JAX-RS
    • None
    • Unknown

    Description

      Concurrent usage of Webclients causes massively more threads to be created in cxf-3.6.x/4.x than before.
      Supposedly this results from introducing a new conduit implementation based on the "new" java http client compared to HttpUrlConnection before.

      It seems that a new http client is created per requesting thread - and each http client in turn creates an own pool of selector/worker threads.

      To demonstate I've attached a test project with several methods to test different scenarios.

      Each tests launches a simple JAX-RS Server, configures/uses WebClients in different configuration and submits 1000 requests via up to 100 parrallel requestor threads.

      The number of live threads in the JVM instance if printed after each request. Baseline is the number of threads used by the server instance + the 100 requesting threads.

       

      singleClientInstanceWithNewConduit -> reuses a threadsafe client instance, thread count rises to 700+ live threads

      singleClientInstanceWithOldConduit -> reuses a threadsafe client instance, stays at about 200 threads

      clientPerRequestWithOldConduit     -> creates a client instance per request (no closing!), keeps below 200 threads

      clientPerRequestWithNewConduit     -> creates a client instance per request (no closing!), creates a runaway thread leak

      clientPerRequestWithNewConduit is additionally interesting because current documentation is not clear about whether client instances
      are required to be resource managed or not.
      It appears that closing clients was not required up until cxf-3.6.0 and documentation actively encourages creating new "lightweight" client instances on the fly (see Thread safety in https://cxf.apache.org/docs/jax-rs-client-api.html).

      However, cxf-3.6+ implementation (and comments in CXF-8885) seem to suggest that clients are in fact not lightweight (anymore?) but need to be closed, when no longer used.
      But then in turn WebClient/Client does not even implement Closeable/Autoclosable. So, it's all quite muddy - and there doesn't seem to be a real consensus on the idiomatic usage of Webclients.
      It would be very nice if you could bring some clarity on this as well.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            reta Andriy Redko
            lkoe Lars Ködderitzsch
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment