ODE
  1. ODE
  2. ODE-533

Share a connection manager across SoapExternalServices

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.1, 1.1.1, 1.2
    • Fix Version/s: 1.3.2, 2.0
    • Component/s: Axis2 Integration
    • Labels:
      None

      Description

      All external service should share the same http connection manager. The connection manager will be initialized by ODEServer. ExternalServices would receive a ConfigurationContext instance containing this connection manager.

      Configuration of the connection manager will be done through ode-axis2.properties :
      "http.connection-manager.max-per-host"
      "http.connection-manager.max-total"

      In addition, we should make sure axis2 release the connections to the pool. By invoking OperationClient#complete.

      see http://markmail.org/thread/voabzcl6u2pck74n

        Issue Links

          Activity

          Alexis Midon created issue -
          Alexis Midon made changes -
          Field Original Value New Value
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          Alexis Midon added a comment -

          committed in 1.X 749508.

          To see all pool activity, activate the log category :org.apache.commons.httpclient.MultiThreadedHttpConnectionManager

          Show
          Alexis Midon added a comment - committed in 1.X 749508. To see all pool activity, activate the log category :org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
          Hide
          Alexis Midon added a comment -

          committed in trunk 749709.

          Show
          Alexis Midon added a comment - committed in trunk 749709.
          Alexis Midon made changes -
          Fix Version/s 2.0 [ 12313503 ]
          Fix Version/s 1.3.1 [ 12313680 ]
          Status In Progress [ 3 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Alexis Midon made changes -
          Link This issue is duplicated by ODE-537 [ ODE-537 ]
          Alexis Midon made changes -
          Status Resolved [ 5 ] Reopened [ 4 ]
          Resolution Fixed [ 1 ]
          Hide
          Ciaran Jessup added a comment -

          Sorry about ODE-537 Matthieu asked for a new bug to be raised (we figured out that the regression came in with your changes ). While you're back in there, those properties don't work either, reading the code it looked as though they should be:
          ode-axis2.http.connection-manager.foo.bar
          Is this deliberate ? (The property bag appears to stick an unexpected prefix in there)

          Show
          Ciaran Jessup added a comment - Sorry about ODE-537 Matthieu asked for a new bug to be raised (we figured out that the regression came in with your changes ). While you're back in there, those properties don't work either, reading the code it looked as though they should be: ode-axis2.http.connection-manager.foo.bar Is this deliberate ? (The property bag appears to stick an unexpected prefix in there)
          Hide
          Alexis Midon added a comment -

          No pb. Thanks for your feedback. Actually I'm glad you rose the issue because I've tried to trigger it but in vain.
          Ill look att your patch today. And fix the issue asap.

          regarding the property names, yes they should be prefixed by ode-axis2. I assumed it implicitly because that's a convention for that file. I apologize for the guesswork you had to do.

          Show
          Alexis Midon added a comment - No pb. Thanks for your feedback. Actually I'm glad you rose the issue because I've tried to trigger it but in vain. Ill look att your patch today. And fix the issue asap. regarding the property names, yes they should be prefixed by ode-axis2. I assumed it implicitly because that's a convention for that file. I apologize for the guesswork you had to do.
          midon committed 749847 (1 file)
          Reviews: none

          ODE-533: not releasing the conn is bad, releasing too early as well

          midon committed 749865 (1 file)
          midon committed 749866 (1 file)
          Hide
          Alexis Midon added a comment -

          take 2: thanks for the patch Colin and Ciaran.

          Show
          Alexis Midon added a comment - take 2: thanks for the patch Colin and Ciaran.
          Alexis Midon made changes -
          Resolution Fixed [ 1 ]
          Status Reopened [ 4 ] Resolved [ 5 ]
          midon committed 753438 (1 file)
          midon committed 753440 (1 file)
          midon committed 763815 (1 file)
          Reviews: none

          ODE-533: by default conn pool max size is the same as thread pool max size

          Hide
          Alexis Midon added a comment -

          In SoapExternalService, the invocation of the service happens in a thread taken out of the common thread pool. This invocation retrieves an HTTP connection from the connection pool. Assuming the max number of connections is small, and that many invocations happen simultaneously, other threads (retrieved from the same pool) will be waiting for a connection. As invocations are piling up, threads are retrieved from the pool and wait for a connection. As a result the thread pool get starved, waiting for a thread to be released.
          Here one could think that this is a regular thread pool starvation scenario and that once a thread will release a connection, a thread will get it, do its job and get back to the pool. Nothing more that jobs sequentialization.

          But actually this case is different because while the service invocation happens in thread A, the release of the HTTP connection takes place in yet another thread, thread B. And guess where Thread B comes from? The exact same thread pool we just starved. So, the connection release never gets a chance to happen. => Engine locked.

          The immediate to fix is to increase the size of the connection pool. Actually the connection pool size should be greater that the thread pool size. That's what r763815 does.

          A better solution would be to eliminate the need for Thread B, and do the connection release in the thread A. ODE-577 tracks this.

          Show
          Alexis Midon added a comment - In SoapExternalService, the invocation of the service happens in a thread taken out of the common thread pool. This invocation retrieves an HTTP connection from the connection pool. Assuming the max number of connections is small, and that many invocations happen simultaneously, other threads (retrieved from the same pool) will be waiting for a connection. As invocations are piling up, threads are retrieved from the pool and wait for a connection. As a result the thread pool get starved, waiting for a thread to be released. Here one could think that this is a regular thread pool starvation scenario and that once a thread will release a connection, a thread will get it, do its job and get back to the pool. Nothing more that jobs sequentialization. But actually this case is different because while the service invocation happens in thread A, the release of the HTTP connection takes place in yet another thread, thread B. And guess where Thread B comes from? The exact same thread pool we just starved. So, the connection release never gets a chance to happen. => Engine locked. The immediate to fix is to increase the size of the connection pool. Actually the connection pool size should be greater that the thread pool size. That's what r763815 does. A better solution would be to eliminate the need for Thread B, and do the connection release in the thread A. ODE-577 tracks this.
          Alexis Midon made changes -
          Link This issue relates to ODE-577 [ ODE-577 ]
          Alexis Midon made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Alexis Midon made changes -
          Fix Version/s 1.3.2 [ 12313906 ]
          Fix Version/s 1.3.1 [ 12313680 ]

            People

            • Assignee:
              Alexis Midon
              Reporter:
              Alexis Midon
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development