[WW-3580] Critical performance issue in production environment as thread dumps are leading to OGNL 3.0 thread blocking! Website could be backed out!

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 2.3.1

Environment

Struts 2.2.1

• Assignee: Lukasz Lenart
• Reporter: Shishir Saxena
• Created: 22/Feb/11 18:46
• Updated: 17/Oct/11 16:18
• Resolved: 10/Oct/11 17:05

Description

My web application based on Struts 2.2.1 is using OGNL 3.0. This web application is rolled into production; however, due to serious performance considerations the website is in danger of being rolled back. The thread dumps indicate 'BLOCKING' at ognl.OgnlRuntime.invokeMethod(OgnlRuntime.java:804).

The thread trace is as:
"httpSSLWorkerThread-6357-6" daemon prio=3 tid=0x01a07000 nid=0xa6 waiting for monitor entry [0xb6d79000..0xb6d7faf0]
java.lang.Thread.State: BLOCKED (on object monitor)
at ognl.OgnlRuntime.invokeMethod(OgnlRuntime.java:804)

• waiting to lock <0xcca6d328> (a java.lang.reflect.Method)
  at ognl.OgnlRuntime.getMethodValue(OgnlRuntime.java:1434)
  at ognl.ObjectPropertyAccessor.getPossibleProperty(ObjectPropertyAccessor.java:60)
  at ognl.ObjectPropertyAccessor.getProperty(ObjectPropertyAccessor.java:147)
  at com.opensymphony.xwork2.ognl.accessor.ObjectAccessor.getProperty(ObjectAccessor.java:17)
  at ognl.OgnlRuntime.getProperty(OgnlRuntime.java:2230)
  at com.opensymphony.xwork2.ognl.accessor.CompoundRootAccessor.getProperty(CompoundRootAccessor.java:137)
  at ognl.OgnlRuntime.getProperty(OgnlRuntime.java:2230)
  at ognl.ASTProperty.getValueBody(ASTProperty.java:114)
  at ognl.SimpleNode.evaluateGetValueBody(SimpleNode.java:212)
  at ognl.SimpleNode.getValue(SimpleNode.java:258)
  at ognl.Ognl.getValue(Ognl.java:494)
  at ognl.Ognl.getValue(Ognl.java:458)
  at com.opensymphony.xwork2.ognl.OgnlUtil.getValue(OgnlUtil.java:213)
  at com.opensymphony.xwork2.ognl.OgnlValueStack.getValueUsingOgnl(OgnlValueStack.java:277)
  at com.opensymphony.xwork2.ognl.OgnlValueStack.tryFindValue(OgnlValueStack.java:260)
  at com.opensymphony.xwork2.ognl.OgnlValueStack.tryFindValueWhenExpressionIsNotNull(OgnlValueStack.java:242)
  at com.opensymphony.xwork2.ognl.OgnlValueStack.findValue(OgnlValueStack.java:222)
  at com.opensymphony.xwork2.ognl.OgnlValueStack.findValue(OgnlValueStack.java:284)
  at sun.reflect.GeneratedMethodAccessor111.invoke(Unknown Source)
  ....

quot;httpSSLWorkerThread-6357-4" daemon prio=3 tid=0x01a56800 nid=0xa4 runnable [0xb6f79000..0xb6f7fbf0]
java.lang.Thread.State: RUNNABLE
at java.security.AccessController.$$YJP$$doPrivileged(Native Method)
at java.security.AccessController.doPrivileged(AccessController.java)
at com.sun.enterprise.security.provider.PolicyFile.addPermissions(PolicyFile.java:1333)
at com.sun.enterprise.security.provider.PolicyFile.getPermissions(PolicyFile.java:1290)
at com.sun.enterprise.security.provider.PolicyFile.getPermissions(PolicyFile.java:1256)
at com.sun.enterprise.security.provider.PolicyFile.getPermissions(PolicyFile.java:1198)
at com.sun.enterprise.security.provider.PolicyFile.implies(PolicyFile.java:1153)
at com.sun.enterprise.security.provider.BasePolicyWrapper.doImplies(BasePolicyWrapper.java:383)
at com.sun.enterprise.security.provider.BasePolicyWrapper.implies(BasePolicyWrapper.java:237)
at java.security.ProtectionDomain.implies(ProtectionDomain.java:213)
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:301)
at java.security.AccessController.checkPermission(AccessController.java:546)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
at java.lang.reflect.AccessibleObject.setAccessible(AccessibleObject.java:107)
at ognl.OgnlRuntime.invokeMethod(OgnlRuntime.java:839)

• locked <0xcca6d328> (a java.lang.reflect.Method)
  at ognl.OgnlRuntime.getMethodValue(OgnlRuntime.java:1434)
  at ognl.ObjectPropertyAccessor.getPossibleProperty(ObjectPropertyAccessor.java:60)

Issue Links

• is blocked by: OGNL-185 Performance issue on high load (thread blocking)
• is blocked by: OGNL-211 Performance - Replace synchronized blocks with ReentrantReadWriteLock