Axis
  1. Axis
  2. AXIS-2647

100% CPU hang in TypeDesc.java:313

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: 1.4
    • Fix Version/s: None
    • Labels:
    • Environment:
      JDK 1.5.0_10 AMD64
      RHEL4 x64

      Description

      Basically we have a bunch of threads stuck within Axis consuming 100% of our CPU. This appears to be during deserialization of a response (from EBay API in this instance). My best guess would be some sort of thread-unsafe modification to the HashMap used by getFieldNameForElement(), resulting in a corrupted internal HashMap state and thusly infinite looping.

      ----------------------------------------------------------
      Name: http-8080-Processor51
      State: RUNNABLE
      Total blocked: 52 Total waited: 1,995

      Stack trace:
      java.util.HashMap.get(HashMap.java:346)
      org.apache.axis.description.TypeDesc.getFieldNameForElement(TypeDesc.java:313)
      org.apache.axis.encoding.ser.BeanDeserializer.onStartChild(BeanDeserializer.java:197)
      org.apache.axis.encoding.DeserializationContext.startElement(DeserializationContext.java:1035)
      org.apache.axis.message.SAX2EventRecorder.replay(SAX2EventRecorder.java:165)
      org.apache.axis.message.MessageElement.publishToHandler(MessageElement.java:1141)
      org.apache.axis.message.RPCElement.deserialize(RPCElement.java:236)
      org.apache.axis.message.RPCElement.getParams(RPCElement.java:384)
      org.apache.axis.client.Call.invoke(Call.java:2467)
      org.apache.axis.client.Call.invoke(Call.java:2366)
      org.apache.axis.client.Call.invoke(Call.java:1812)
      com.automotive.app.content.ebay.api.EBayAPISoapBindingStub.getItem(EBayAPISoapBindingStub.java:7962)
      .....

      Name: http-8080-Processor25
      State: RUNNABLE
      Total blocked: 295 Total waited: 10,588

      Stack trace:
      java.util.HashMap.get(HashMap.java:346)
      org.apache.axis.description.TypeDesc.getFieldNameForElement(TypeDesc.java:313)
      org.apache.axis.encoding.ser.BeanDeserializer.onStartChild(BeanDeserializer.java:197)
      org.apache.axis.encoding.DeserializationContext.startElement(DeserializationContext.java:1035)
      org.apache.axis.message.SAX2EventRecorder.replay(SAX2EventRecorder.java:165)
      org.apache.axis.message.MessageElement.publishToHandler(MessageElement.java:1141)
      org.apache.axis.message.RPCElement.deserialize(RPCElement.java:236)
      org.apache.axis.message.RPCElement.getParams(RPCElement.java:384)
      org.apache.axis.client.Call.invoke(Call.java:2467)
      org.apache.axis.client.Call.invoke(Call.java:2366)
      org.apache.axis.client.Call.invoke(Call.java:1812)
      com.automotive.app.content.ebay.api.EBayAPISoapBindingStub.getItem(EBayAPISoapBindingStub.java:7962)
      .....

      1. TypeDesc.java
        17 kB
        Andrey Adamovich

        Activity

        Hide
        Patrick Eger added a comment -

        Researching further, this is a definite thread safety problem in TypeDesc, specifically the fields fieldNameMap, fieldElementMap are accessed in a thread-unsafe manner in many functions, setFields(), getFieldNameForElement(), addFieldDesc(), getElementNameForField(), getAttributeNameForField(), getFieldByName(). I don't know enough about the engine to say whether the fieldNameMap can be accessed concurrently, but fieldElementMap obviously can be and is clearly broken.

        Show
        Patrick Eger added a comment - Researching further, this is a definite thread safety problem in TypeDesc, specifically the fields fieldNameMap, fieldElementMap are accessed in a thread-unsafe manner in many functions, setFields(), getFieldNameForElement(), addFieldDesc(), getElementNameForField(), getAttributeNameForField(), getFieldByName(). I don't know enough about the engine to say whether the fieldNameMap can be accessed concurrently, but fieldElementMap obviously can be and is clearly broken.
        Hide
        Andrey Adamovich added a comment -

        We faced the same issue and fixed that by patching Axis code i.e. putting syncronisation blocks inside TypeDesc class to secure access to fieldElementMap.

        Show
        Andrey Adamovich added a comment - We faced the same issue and fixed that by patching Axis code i.e. putting syncronisation blocks inside TypeDesc class to secure access to fieldElementMap.
        Hide
        Le Huy added a comment -

        We are facing similar problem, in which getFieldNameForElement entered an infinite loop due to corruption if the hashmap used as cache without proper synchronization.
        Can someone share the patch that synchronize access inside TypeDesc class

        Show
        Le Huy added a comment - We are facing similar problem, in which getFieldNameForElement entered an infinite loop due to corruption if the hashmap used as cache without proper synchronization. Can someone share the patch that synchronize access inside TypeDesc class
        Hide
        Andrey Adamovich added a comment -

        This is the file we used to patch our version of axis.

        Show
        Andrey Adamovich added a comment - This is the file we used to patch our version of axis.

          People

          • Assignee:
            Unassigned
            Reporter:
            Patrick Eger
          • Votes:
            2 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development