Ivy
  1. Ivy
  2. IVY-1325

ivy:report ant task intermittently "cannot compile stylesheet"

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0-RC1
    • Component/s: None
    • Labels:
      None
    • Environment:

      ivy-2.2.jar, Sun jdk 1.5 and 1.6 for compiling, Jenkins 1.436 as CI server, ant 1.8.2, Linux, Jenkins ivy plugin 1.20

      Description

      Setup:

      o Using ivy-2.2.jar.

      o Compiling with Sun jdk versions 1.5 or 1.6 (project dependent) on a Linux box.

      o Running ant 1.8.2 via a wrapper script inside Jenkins 1.436 with 5 or so
      executors. That is, each build invokes a simple custom wrapper-script around
      Ant that puts some custom tasks and ivy-2.2 on the build classpath.

      o Jenkins Ivy plugin version is version 1.20 which uses ivy 2.2.0 internally.

      Problem:

      We hit this error perhaps 8 % of the builds when there are several
      parallel builds happening:

      javax.xml.transform.TransformerConfigurationException: Could not compile stylesheet
      at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.newTemplates(TransformerFactoryImpl.java:828)
      at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.newTransformer(TransformerFactoryImpl.java:617)
      at org.apache.ivy.ant.IvyReport.genStyled(IvyReport.java:322)
      at org.apache.ivy.ant.IvyReport.genreport(IvyReport.java:248)
      at org.apache.ivy.ant.IvyReport.doExecute(IvyReport.java:211)
      at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
      :
      :

      The build always runs fine for a single job when invoked from the
      command line, and 99.9% of the time runs fine when the Jenkins job is
      requeued. But when it happens during parallel builds, the error
      message is always the same and always in exactly the same spot in
      IvyReport.java.

      The relevant ant task is this (we are behind a firewall so retrieves
      never go out to the internet, all resolvers are FileSystem resolvers,
      which explains some of the attribute values):

      <macrodef name="resolve-dependencies">
      <attribute name="ivyconfig"/>
      <sequential>
      <ivy:resolve
      conf="@

      {ivyconfig}"
      log="${option.ivy.log}"
      resolveMode="dynamic"
      showprogress="false"
      checkIfChanged="false"
      />

      <ivy:retrieve
      conf="@{ivyconfig}

      "
      log="$

      {option.ivy.log}

      "
      pattern="$

      {ivy.lib.dir}

      /@

      {ivyconfig}/[artifact](-[revision]).[ext]"
      symlink="true"
      sync="true"
      />

      <ivy:report <---------------- INTERMITTENT FAILURE DURING THIS TASK
      conf="@{ivyconfig}

      "
      dot="$

      {option.ivyreport.dot}

      " <--- false on Jenkins, settable for indiv developer
      graph="$

      {option.ivyreport.graphml}

      " <--- false on Jenkins, settable for indiv developer
      outputpattern="dependencies-[conf].[ext]"
      todir="$

      {dir.build}

      /report"
      />
      </sequential>
      </macrodef>

      As you can see from the code, we are not using our own stylesheet, but
      the one in the ivy distribution.

      The code in IvyReport.java looks ok to me but I am not a TRaX user and
      certainly no expert on it so it is not clear to me why occasionally we
      get this "cannot compile stylesheet" error. And we only get it when
      many builds are happening in parallel, never for one-build-at-a-time,
      which makes me wonder if there is some issue with grabbing resources
      from inside a jar, something I do not do myself.

        Activity

        Hide
        David Goblirsch added a comment -

        I made 2 changes which seem to reduce the frequency
        with which this problem occurs.

        1) First, I switched from using

        ivy:settings
        to
        ivy:configure

        and it seemed to happen somewhat less often. But this is not a
        rigorous observation. Since the error seems random, occurs
        infrequently, and only when Jenkins is really, really busy with
        parallel builds, it is hard to collect an accurate frequency of
        occurrence.

        2) I changed to specifying that we wanted the JVM running
        ant to use

        xalan-2.7.1 and xerces-2.9.1.

        instead of the built-in parsers and transformers. The error still
        happens on the ivy:report task, but the trace is slighty different:

        :
        [ivy:report] [Fatal Error] ivy-report.xsl:196:8: XML document structures must start and end within the same entity.
        [ivy:report] file:/home/users/hudson/.ivy2/cache/ivy-report.xsl; Line #196; Column #8; org.xml.sax.SAXParseException:
        XML document structures must start and end within the same entity.
        :

        macros.xml:87: java.lang.NullPointerException
        at org.apache.xalan.transformer.TransformerImpl.setParameter(TransformerImpl.java:1547)
        at org.apache.xalan.transformer.TransformerImpl.setParameter(TransformerImpl.java:1595)
        at org.apache.ivy.ant.IvyReport.genStyled(IvyReport.java:325)
        at org.apache.ivy.ant.IvyReport.genreport(IvyReport.java:248)
        at org.apache.ivy.ant.IvyReport.doExecute(IvyReport.java:211)
        at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        :
        :

        Here, the macro being excuted is this (excised from
        a somewhat larger file):

        <macrodef name="resolve-dependencies">
        <attribute name="ivyconfig"/>
        <sequential>
        <ivy:resolve
        conf="@

        {ivyconfig}"
        log="${option.ivy.log}"
        resolveMode="dynamic"
        showprogress="false"
        checkIfChanged="false"
        />

        <ivy:retrieve
        conf="@{ivyconfig}

        "
        log="$

        {option.ivy.log}

        "
        pattern="$

        {ivy.lib.dir}

        /@

        {ivyconfig}/[artifact](-[revision]).[ext]"
        symlink="true"
        sync="true"
        />

        <ivy:report
        conf="@{ivyconfig}

        "
        dot="$

        {option.ivyreport.dot}

        "
        graph="$

        {generate-ivyreport-graphml}

        "
        outputpattern="dependencies-[conf].[ext]"
        todir="$

        {dir.build}

        /report"
        /> <============ Line 87 of the complete macros file

        </sequential>
        </macrodef>

        With these two changes, the frequency of occurrence seems to be
        reduced. But it still happens.

        Again, the error is never observed on individual developer runs, or
        even when there is low Jenkins activity. Only occasionally when there
        is a large queue of builds keeping Jenkins executors fully busy.

        We are on a private network, so the resolvers are all FileSystem
        resolvers.

        QUESTION FOR THE EXPERTS: Could it be that I need to use transactions
        on the FileSystem resolvers? How would that be related to XSLT errors?

        Show
        David Goblirsch added a comment - I made 2 changes which seem to reduce the frequency with which this problem occurs. 1) First, I switched from using ivy:settings to ivy:configure and it seemed to happen somewhat less often. But this is not a rigorous observation. Since the error seems random, occurs infrequently, and only when Jenkins is really, really busy with parallel builds, it is hard to collect an accurate frequency of occurrence. 2) I changed to specifying that we wanted the JVM running ant to use xalan-2.7.1 and xerces-2.9.1. instead of the built-in parsers and transformers. The error still happens on the ivy:report task, but the trace is slighty different: : [ivy:report] [Fatal Error] ivy-report.xsl:196:8: XML document structures must start and end within the same entity. [ivy:report] file:/home/users/hudson/.ivy2/cache/ivy-report.xsl ; Line #196; Column #8; org.xml.sax.SAXParseException: XML document structures must start and end within the same entity. : macros.xml:87: java.lang.NullPointerException at org.apache.xalan.transformer.TransformerImpl.setParameter(TransformerImpl.java:1547) at org.apache.xalan.transformer.TransformerImpl.setParameter(TransformerImpl.java:1595) at org.apache.ivy.ant.IvyReport.genStyled(IvyReport.java:325) at org.apache.ivy.ant.IvyReport.genreport(IvyReport.java:248) at org.apache.ivy.ant.IvyReport.doExecute(IvyReport.java:211) at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) : : Here, the macro being excuted is this (excised from a somewhat larger file): <macrodef name="resolve-dependencies"> <attribute name="ivyconfig"/> <sequential> <ivy:resolve conf="@ {ivyconfig}" log="${option.ivy.log}" resolveMode="dynamic" showprogress="false" checkIfChanged="false" /> <ivy:retrieve conf="@{ivyconfig} " log="$ {option.ivy.log} " pattern="$ {ivy.lib.dir} /@ {ivyconfig}/ [artifact] (- [revision] ). [ext] " symlink="true" sync="true" /> <ivy:report conf="@{ivyconfig} " dot="$ {option.ivyreport.dot} " graph="$ {generate-ivyreport-graphml} " outputpattern="dependencies- [conf] . [ext] " todir="$ {dir.build} /report" /> <============ Line 87 of the complete macros file </sequential> </macrodef> With these two changes, the frequency of occurrence seems to be reduced. But it still happens. Again, the error is never observed on individual developer runs, or even when there is low Jenkins activity. Only occasionally when there is a large queue of builds keeping Jenkins executors fully busy. We are on a private network, so the resolvers are all FileSystem resolvers. QUESTION FOR THE EXPERTS: Could it be that I need to use transactions on the FileSystem resolvers? How would that be related to XSLT errors?
        Hide
        Maarten Coene added a comment -

        Could you try again with Ivy trunk?
        I've just committed something that maybe could fix the problem.

        Show
        Maarten Coene added a comment - Could you try again with Ivy trunk? I've just committed something that maybe could fix the problem.
        Hide
        David Goblirsch added a comment -

        Yes, thank you Maarten, that fixed the problem. I can now
        run heavy parallel builds and have not seen the error
        recur since moving to 2.2.1.alpha_20111225190156.

        Thanks so much!

        Show
        David Goblirsch added a comment - Yes, thank you Maarten, that fixed the problem. I can now run heavy parallel builds and have not seen the error recur since moving to 2.2.1.alpha_20111225190156. Thanks so much!

          People

          • Assignee:
            Maarten Coene
            Reporter:
            David Goblirsch
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development