Abdera
  1. Abdera
  2. ABDERA-267

Major performance issue with Abdera (underlying Axiom) object model while writing Atom DOM to XmlStreamWriter

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.1
    • Fix Version/s: 1.1.1
    • Labels:
      None
    • Environment:
      N/A

      Description

      Background:
      Abdera object model (OM) is based off of Axiom OM. In FOMDocument class (which extends Axiom's OMDocumentImpl), method toWrite(java.io.Writer) makes a call to this.internalSerialize(javax.xml.stream.XmlStreamWriter). After this point Abdera delegate the XML stream writing to Axiom.

      Issue:
      Axiom 1.2.7, has a serious performance issue if one is using likes of woodstox's implementation (com.ctc.wstx.sw.SimpleNsStreamWriter - http://woodstox.codehaus.org/3.2.9/javadoc/index.html ), which I believe is the default writer used by Axiom.

      The internalSerialize call eventually makes it to org.apache.axiom.om.impl.util.OMSerializerUtil.serializeStartpart(OMElement element, String localName, XMLStreamWriter writer). Now, here starts the major performance hog. This makes a call to isSetPrefixBeforeStartElement(XmlStreamWriter), which handles one of the MOST controversial part of the XmlStreamWriter spec - http://download.oracle.com/javase/6/docs/api/javax/xml/stream/XMLStreamWriter.html.

      They fairly recognized this in the javadocs for OMSerializerUtil.isSetPrefixBeforeStartElement in Axiom version 1.2.7. But they got the implementation all wrong. The implementation looks for the property javax.xml.stream.XMLStreamWriter.isSetPrefixBeforeStartElement in the writer, which throws an IllegalArgumentException if this property is not found, which IS NOT FOUND. The implementation then catches this exception and returns false. This happens for EVERY single element written to output stream. This exception handling is way too much expensive and when it happens per element in XML per request, it takes MOST part of the processing time every request. More on this specific property - http://publib.boulder.ibm.com/infocenter/realtime/v2r0/index.jsp?topic=/com.ibm.rt.doc.20/user/xml/xlxpj_reference.html.

      Since Abdera is meant for Atom feed, when there are concurrent requests the overall latency per request increases more than linearly and after a while it becomes unusable for high load.

      Axiom realized this flaw in their logic and fixed it in 1.2.9. Hence, Abdera SHOULD consider upgrading it's code to use Axiom 1.2.9 instead of 1.2.7, and this performance issue will be gone. For our use, I explicitly made this change to use the new Axiom library but there are other dependencies which is preventing me to make the upgrade that easily.

      Most importantly, after upgrading the JAR, I get the following exception.:

      Caused by: java.lang.IllegalStateException: This factory is immutable
      at org.apache.axiom.util.stax.wrapper.ImmutableXMLOutputFactory.setProperty(ImmutableXMLOutputFactory.java:39)
      at org.apache.abdera.parser.stax.StaxStreamWriter.createXMLStreamWriter(StaxStreamWriter.java:106)
      at org.apache.abdera.parser.stax.StaxStreamWriter.setOutputStream(StaxStreamWriter.java:113)

      This is because javax.xml.stream.XmlOutputFactory and javax.xml.stream.XmlInputFactory, that Axiom 1.2.9 creates, are now immutable and Abdera code is trying to set some javax.xml.stream.* properties in them. To fix this, I updated StaxStreamWriter.createXMLStreamWriter and FOMParser.getXMLInputFactory to NOT set any property. For our purposes this seems to work fine so far, but perhaps needs a bit more research to make the real fix.

      Action items:
      1. Upgrade to Axiom 1.2.9
      2. Fix places where Abdera (esp. parser module) is trying to modify the immutable XmlInputFactory and XmlOutputFactory as returned by Axiom.

      1. ABDERA-267.patch
        9 kB
        Andreas Veithen

        Issue Links

          Activity

          Hide
          ant elder added a comment -

          Thanks for the detailed problem report Abhishek, this does sound like a useful update to do.

          I had a try at updating Abdera to use axiom 1.2.9 and found there are a couple of non-backward compatible API changes so we get three compile errors in the Abdera FOMDocument class all related to constructor changes in axiom OMDocumentImpl. I don't suppose you would care to have a go at tracking down the necessary code changes to getting Abdera to compile cleanly with axiom 1.2.9 would you, and submitting a patch?

          Show
          ant elder added a comment - Thanks for the detailed problem report Abhishek, this does sound like a useful update to do. I had a try at updating Abdera to use axiom 1.2.9 and found there are a couple of non-backward compatible API changes so we get three compile errors in the Abdera FOMDocument class all related to constructor changes in axiom OMDocumentImpl. I don't suppose you would care to have a go at tracking down the necessary code changes to getting Abdera to compile cleanly with axiom 1.2.9 would you, and submitting a patch?
          Hide
          Abhishek Shadangi added a comment -

          Thanks for going over this issues and giving more insight into the issues with Axiom upgrade. Definitely, I will be happy to investigate further. I might not be able to get to it right away but I will try to look into it next week and update this issue here ASAP.

          Show
          Abhishek Shadangi added a comment - Thanks for going over this issues and giving more insight into the issues with Axiom upgrade. Definitely, I will be happy to investigate further. I might not be able to get to it right away but I will try to look into it next week and update this issue here ASAP.
          Hide
          Andreas Veithen added a comment -

          Abhishek's analysis is correct, and I would like to add some more explanations (wearing my Axiom developer hat):

          • Some background information about the isSetPrefixBeforeStartElement thing can be found in [1]. It is actually not Axiom that got it wrong, but a particular version of a particular StAX implementation. In Axiom 1.2.9, we isolated that code into what we call a StAX dialect so that it is only ever triggered when that particular StAX implementation is used.
          • On the other hand, the performance issue related to isSetPrefixBeforeStartElement was already addressed in 1.2.8 (see WSCOMMONS-325). So upgrading to that version instead of 1.2.9 may provide a quick fix for the issue.
          • In Axiom 1.2.9, the factories returned by StAXUtils are indeed immutable. The reason is that these factories are cached and therefore shared with other frameworks using Axiom. E.g. if Abdera and Axis2 are used in the same application, they would get the same factory instances from Axiom. Therefore we can't allow modifications of these factory instances, because this has the potential of causing subtle issues. In 1.2.9, we also introduced new features that allow to customize the factories in a controlled way, but for XMLOutputFactory instances, this is only implemented in the current trunk.
          • Since Abdera extends Axiom implementation classes (namely the LLOM implementation), it is indeed strongly coupled and therefore very sensitive to changes in Axiom. Probably the best way to avoid these issues is to have the Abdera trunk depend on the Axiom trunk so that incompatibilities are detected early by the Abdera build in Hudson.

          I will see what needs to be done in the Abdera and Axiom code to make them again play nicely together.

          [1] http://markmail.org/thread/37y3qjcfucb6756o

          Show
          Andreas Veithen added a comment - Abhishek's analysis is correct, and I would like to add some more explanations (wearing my Axiom developer hat): Some background information about the isSetPrefixBeforeStartElement thing can be found in [1] . It is actually not Axiom that got it wrong, but a particular version of a particular StAX implementation. In Axiom 1.2.9, we isolated that code into what we call a StAX dialect so that it is only ever triggered when that particular StAX implementation is used. On the other hand, the performance issue related to isSetPrefixBeforeStartElement was already addressed in 1.2.8 (see WSCOMMONS-325 ). So upgrading to that version instead of 1.2.9 may provide a quick fix for the issue. In Axiom 1.2.9, the factories returned by StAXUtils are indeed immutable. The reason is that these factories are cached and therefore shared with other frameworks using Axiom. E.g. if Abdera and Axis2 are used in the same application, they would get the same factory instances from Axiom. Therefore we can't allow modifications of these factory instances, because this has the potential of causing subtle issues. In 1.2.9, we also introduced new features that allow to customize the factories in a controlled way, but for XMLOutputFactory instances, this is only implemented in the current trunk. Since Abdera extends Axiom implementation classes (namely the LLOM implementation), it is indeed strongly coupled and therefore very sensitive to changes in Axiom. Probably the best way to avoid these issues is to have the Abdera trunk depend on the Axiom trunk so that incompatibilities are detected early by the Abdera build in Hudson. I will see what needs to be done in the Abdera and Axiom code to make them again play nicely together. [1] http://markmail.org/thread/37y3qjcfucb6756o
          Hide
          Andreas Veithen added a comment -

          Attached a patch that updates the trunk (r1005010) to Axiom 1.2.10-SNAPSHOT.

          Please let me know what are the plans for the Abdera 1.1 release, so that we can get Axiom 1.2.10 out on time.

          Show
          Andreas Veithen added a comment - Attached a patch that updates the trunk (r1005010) to Axiom 1.2.10-SNAPSHOT. Please let me know what are the plans for the Abdera 1.1 release, so that we can get Axiom 1.2.10 out on time.
          Hide
          Abhishek Shadangi added a comment - - edited

          Thanks Andreas for giving Axiom developer's perspective Good to learn further about the real cause. I realized the rationale behind immutability of XMLInputFactory and XMLOutputFactory when I thought more about it. Thanks for clarifying though.

          I was about to upload the EXACT SAME patch here. This will work as there is caching per configuration per classloader.

          Show
          Abhishek Shadangi added a comment - - edited Thanks Andreas for giving Axiom developer's perspective Good to learn further about the real cause. I realized the rationale behind immutability of XMLInputFactory and XMLOutputFactory when I thought more about it. Thanks for clarifying though. I was about to upload the EXACT SAME patch here. This will work as there is caching per configuration per classloader.
          Hide
          ant elder added a comment -

          Thanks both of you for for working on this issue, i've applied the patch to Abdera trunk in r1006114.
          We need to get this and a couple of other fixes out in an Abdera release but wont be able to do that till an Axiom release is done so Abdera can used a non-snapshot release. When do you think an Axiom release is likely to happen?
          (lets leave this jira open till the release is done)

          Show
          ant elder added a comment - Thanks both of you for for working on this issue, i've applied the patch to Abdera trunk in r1006114. We need to get this and a couple of other fixes out in an Abdera release but wont be able to do that till an Axiom release is done so Abdera can used a non-snapshot release. When do you think an Axiom release is likely to happen? (lets leave this jira open till the release is done)
          Hide
          Andreas Veithen added a comment -

          There are currently two JIRAs (WSCOMMONS-556 and WSCOMMONS-562) that need to be completed before the next Axiom release. It should not take more than a week to finish them. Since for the 1.2.9 release I've streamlined the release process, you could expect a 1.2.10 release available in the Maven repositories before the end of the month.

          Show
          Andreas Veithen added a comment - There are currently two JIRAs ( WSCOMMONS-556 and WSCOMMONS-562 ) that need to be completed before the next Axiom release. It should not take more than a week to finish them. Since for the 1.2.9 release I've streamlined the release process, you could expect a 1.2.10 release available in the Maven repositories before the end of the month.
          Hide
          ant elder added a comment -

          Terrific, that sounds fine.

          Show
          ant elder added a comment - Terrific, that sounds fine.
          Hide
          Andreas Veithen added a comment -

          The code on the Axiom trunk is now in a releasable state. On the other hand, there are some ongoing changes to the structure of the Web Services project (mailing list changes, disassembling of the 'commons' notion) that need to be reflected in the POM metadata and the Maven site. Once the dust has settled and I have all the information (this should take a couple of days), I can do the necessary updates and start the release process.

          Show
          Andreas Veithen added a comment - The code on the Axiom trunk is now in a releasable state. On the other hand, there are some ongoing changes to the structure of the Web Services project (mailing list changes, disassembling of the 'commons' notion) that need to be reflected in the POM metadata and the Maven site. Once the dust has settled and I have all the information (this should take a couple of days), I can do the necessary updates and start the release process.
          Hide
          Andreas Veithen added a comment -

          The vote for the Axiom 1.2.10 release is now open on dev@ws.apache.org.

          Show
          Andreas Veithen added a comment - The vote for the Axiom 1.2.10 release is now open on dev@ws.apache.org.
          Hide
          ant elder added a comment -

          Axiom 1.2.10 is now out so we should do an Abdera release. Theres a couple of other JIRAs with patches we need to apply and the OSGi issue thats just be raised on the dev list so once they're all done lets do a new Abdera release.

          Show
          ant elder added a comment - Axiom 1.2.10 is now out so we should do an Abdera release. Theres a couple of other JIRAs with patches we need to apply and the OSGi issue thats just be raised on the dev list so once they're all done lets do a new Abdera release.

            People

            • Assignee:
              Unassigned
              Reporter:
              Abhishek Shadangi
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development