Axis2
  1. Axis2
  2. AXIS2-5118

In high load scenarios with many threads, the call to Introspector.getBeanInfo(Class,Class) causes high synchronization resulting in system stall

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 1.5.4
    • Fix Version/s: 1.6.2, 1.7.0
    • Component/s: adb
    • Labels:
      None
    • Environment:
      Tomcat 7 with axis2 webservices

      Description

      I have a high load scenario, where a Clustered Web frontend does Webservice calls to a Backend Webservices.
      There are around 200 Threads handling the webservices.
      The CPU load of the machine went to near zero.
      Using dynatrace, we realized, that all the calls went through BeanUtils.getPropertyQnameList, which in turn calls Introspector.getBeanInfo(beanClass, beanClass.getSuperclass());
      This again calls WebAppClassLoader.loadClass(String name, boolean resolve), which is synchronized.

      I fixed this problem by modifying BeanUtil.getPropertyQnameList so that it caches BeanInfo objects.

      1. BeanInfoCacheTest.java
        2 kB
        Ronald Brindl
      2. BeanUtil.java
        39 kB
        Ronald Brindl
      3. BeanInfoCache.java
        1 kB
        Ronald Brindl

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Axis2 #1085 (See https://builds.apache.org/job/Axis2/1085/)
          AXIS2-4524 / AXIS2-4878 / AXIS2-5118 / AXIS2-5119: Also implement BeanInfoCachingClassLoader on JarFileClassLoader so that the cache works in all deployment scenarios.

          veithen :
          Files :

          • /axis/axis2/java/core/trunk/modules/kernel/src/org/apache/axis2/classloader/JarFileClassLoader.java
          Show
          Hudson added a comment - Integrated in Axis2 #1085 (See https://builds.apache.org/job/Axis2/1085/ ) AXIS2-4524 / AXIS2-4878 / AXIS2-5118 / AXIS2-5119 : Also implement BeanInfoCachingClassLoader on JarFileClassLoader so that the cache works in all deployment scenarios. veithen : Files : /axis/axis2/java/core/trunk/modules/kernel/src/org/apache/axis2/classloader/JarFileClassLoader.java
          Hide
          Hudson added a comment -

          Integrated in axis2-1.6 #141 (See https://builds.apache.org/job/axis2-1.6/141/)
          AXIS2-4524 / AXIS2-4878 / AXIS2-5118 / AXIS2-5119: Merged r1166132 and r1190499 to the 1.6 branch.

          veithen :
          Files :

          • /axis/axis2/java/core/branches/1_6
          • /axis/axis2/java/core/branches/1_6/modules/adb/src/org/apache/axis2/databinding/utils/BeanUtil.java
          • /axis/axis2/java/core/branches/1_6/modules/kernel/src/org/apache/axis2/classloader/BeanInfoCache.java
          • /axis/axis2/java/core/branches/1_6/modules/kernel/src/org/apache/axis2/classloader/BeanInfoCachingClassLoader.java
          • /axis/axis2/java/core/branches/1_6/modules/kernel/src/org/apache/axis2/classloader/JarFileClassLoader.java
          • /axis/axis2/java/core/branches/1_6/modules/kernel/src/org/apache/axis2/deployment/DeploymentClassLoader.java
          Show
          Hudson added a comment - Integrated in axis2-1.6 #141 (See https://builds.apache.org/job/axis2-1.6/141/ ) AXIS2-4524 / AXIS2-4878 / AXIS2-5118 / AXIS2-5119 : Merged r1166132 and r1190499 to the 1.6 branch. veithen : Files : /axis/axis2/java/core/branches/1_6 /axis/axis2/java/core/branches/1_6/modules/adb/src/org/apache/axis2/databinding/utils/BeanUtil.java /axis/axis2/java/core/branches/1_6/modules/kernel/src/org/apache/axis2/classloader/BeanInfoCache.java /axis/axis2/java/core/branches/1_6/modules/kernel/src/org/apache/axis2/classloader/BeanInfoCachingClassLoader.java /axis/axis2/java/core/branches/1_6/modules/kernel/src/org/apache/axis2/classloader/JarFileClassLoader.java /axis/axis2/java/core/branches/1_6/modules/kernel/src/org/apache/axis2/deployment/DeploymentClassLoader.java
          Hide
          Hudson added a comment -

          Integrated in Axis2 #978 (See https://builds.apache.org/job/Axis2/978/)
          AXIS2-4524 / AXIS2-4878 / AXIS2-5118 / AXIS2-5119: Initial implementation of a BeanInfo cache to speed up POJO services.

          veithen :
          Files :

          • /axis/axis2/java/core/trunk/modules/adb/src/org/apache/axis2/databinding/utils/BeanUtil.java
          • /axis/axis2/java/core/trunk/modules/kernel/src/org/apache/axis2/classloader/BeanInfoCache.java
          • /axis/axis2/java/core/trunk/modules/kernel/src/org/apache/axis2/classloader/BeanInfoCachingClassLoader.java
          • /axis/axis2/java/core/trunk/modules/kernel/src/org/apache/axis2/deployment/DeploymentClassLoader.java
          Show
          Hudson added a comment - Integrated in Axis2 #978 (See https://builds.apache.org/job/Axis2/978/ ) AXIS2-4524 / AXIS2-4878 / AXIS2-5118 / AXIS2-5119 : Initial implementation of a BeanInfo cache to speed up POJO services. veithen : Files : /axis/axis2/java/core/trunk/modules/adb/src/org/apache/axis2/databinding/utils/BeanUtil.java /axis/axis2/java/core/trunk/modules/kernel/src/org/apache/axis2/classloader/BeanInfoCache.java /axis/axis2/java/core/trunk/modules/kernel/src/org/apache/axis2/classloader/BeanInfoCachingClassLoader.java /axis/axis2/java/core/trunk/modules/kernel/src/org/apache/axis2/deployment/DeploymentClassLoader.java
          Hide
          Ronald Brindl added a comment -

          Looks like i have not searched hard enough before entering this issue.
          I understand the concerns in AXIS2-4524. However, they are no issue for us, since the application involved is a demo application for load Testing, which gets restarted by CI completely, anyway.
          Nevertheless, I am looking forward to a generic solution. Integration of the cache into some Webapp-scoped context looks promising!

          Show
          Ronald Brindl added a comment - Looks like i have not searched hard enough before entering this issue. I understand the concerns in AXIS2-4524 . However, they are no issue for us, since the application involved is a demo application for load Testing, which gets restarted by CI completely, anyway. Nevertheless, I am looking forward to a generic solution. Integration of the cache into some Webapp-scoped context looks promising!
          Hide
          Andreas Veithen added a comment -

          This is basically a duplicate of AXIS2-4524 and AXIS2-4878, and the same concern applies here: implementing the cache like that will result in a class loader leak.

          Show
          Andreas Veithen added a comment - This is basically a duplicate of AXIS2-4524 and AXIS2-4878 , and the same concern applies here: implementing the cache like that will result in a class loader leak.
          Hide
          Ronald Brindl added a comment -

          I understand that the call to BeanInfoCache is not 100% thread safe, since it might happen, that between getting null from cache, another thread could do so as well before the cache is filled with the Beaninfor. This might result in a race condition. I consider this a minor risk, the worst that can happen is that the BeanInfo is fetched twice.
          I did heavy load tests with 500 concurrent virtual users running complex Scenarios and did not see any problem there.

          Show
          Ronald Brindl added a comment - I understand that the call to BeanInfoCache is not 100% thread safe, since it might happen, that between getting null from cache, another thread could do so as well before the cache is filled with the Beaninfor. This might result in a race condition. I consider this a minor risk, the worst that can happen is that the BeanInfo is fetched twice. I did heavy load tests with 500 concurrent virtual users running complex Scenarios and did not see any problem there.
          Hide
          Ronald Brindl added a comment - - edited

          Added BeanInfoCache.java, which caches BeanInfo Objects using a limited linked list, which removes oldest entries based on maximum size (currently MAX_SIZE = 10).
          It is thread safe using a ReentrandReadWrite Lock for accessing its backing list.

          In BeanInfo i replaced the call

          BeanInfo beanInfo = Introspector.getBeanInfo(beanClass, beanClass.getSuperclass());

          with

          BeanInfo beanInfo = BeanInfoCache.get(beanClass);
          if(beanInfo == null)

          { beanInfo = Introspector.getBeanInfo(beanClass, beanClass.getSuperclass()); BeanInfoCache.put(beanClass, beanInfo); }
          Show
          Ronald Brindl added a comment - - edited Added BeanInfoCache.java, which caches BeanInfo Objects using a limited linked list, which removes oldest entries based on maximum size (currently MAX_SIZE = 10). It is thread safe using a ReentrandReadWrite Lock for accessing its backing list. In BeanInfo i replaced the call BeanInfo beanInfo = Introspector.getBeanInfo(beanClass, beanClass.getSuperclass()); with BeanInfo beanInfo = BeanInfoCache.get(beanClass); if(beanInfo == null) { beanInfo = Introspector.getBeanInfo(beanClass, beanClass.getSuperclass()); BeanInfoCache.put(beanClass, beanInfo); }

            People

            • Assignee:
              Andreas Veithen
              Reporter:
              Ronald Brindl
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development