Uploaded image for project: 'FOP'
  1. FOP
  2. FOP-2874

[PATCH] FOP conserve memory policy fails in multi-threaded environment

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.2, 2.3
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Flags:
      Patch

      Description

      When conserve memory policy is enabledĀ 
      Code Snippet 1:

      foUserAgent.setConserveMemoryPolicy(true);
      

      in multi-threaded environment, it throws following exception

      SEVERE: Error while serializing page 154. Reason: java.io.IOException: \tmp\fop-page-153.ser5056538250865196860.fop.tmp has been already created for /fop-page-153.ser
      java.io.IOException: \tmp\fop-page-153.ser5056538250865196860.fop.tmp has been already created for /fop-page-153.ser
      	at org.apache.fop.apps.io.ResourceResolverFactory$DefaultTempResourceResolver.createTempFile(ResourceResolverFactory.java:187)
      	at org.apache.fop.apps.io.ResourceResolverFactory$DefaultTempResourceResolver.getOutputStream(ResourceResolverFactory.java:199)
      	at org.apache.fop.apps.io.ResourceResolverFactory$TempAwareResourceResolver.getOutputStream(ResourceResolverFactory.java:159)
      	at org.apache.fop.apps.io.ResourceResolverFactory$DefaultResourceResolver.getOutputStream(ResourceResolverFactory.java:126)
      	at org.apache.fop.apps.io.InternalResourceResolver.getOutputStream(InternalResourceResolver.java:103)
      	at org.apache.fop.area.CachedRenderPagesModel.savePage(CachedRenderPagesModel.java:132)
      	at org.apache.fop.area.CachedRenderPagesModel.checkPreparedPages(CachedRenderPagesModel.java:112)
      	at org.apache.fop.area.RenderPagesModel.addPage(RenderPagesModel.java:146)
      	at org.apache.fop.layoutmgr.AbstractPageSequenceLayoutManager.finishPage(AbstractPageSequenceLayoutManager.java:316)
      	at org.apache.fop.layoutmgr.PageSequenceLayoutManager.finishPage(PageSequenceLayoutManager.java:243)
      	at org.apache.fop.layoutmgr.AbstractPageSequenceLayoutManager.makeNewPage(AbstractPageSequenceLayoutManager.java:287)
      	at org.apache.fop.layoutmgr.PageSequenceLayoutManager.makeNewPage(PageSequenceLayoutManager.java:192)
      	at org.apache.fop.layoutmgr.PageBreaker.handleBreakTrait(PageBreaker.java:633)
      	at org.apache.fop.layoutmgr.PageBreaker.startPart(PageBreaker.java:511)
      	at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:659)
      	at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:604)
      	at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:599)
      	at org.apache.fop.layoutmgr.PageBreaker.doPhase3(PageBreaker.java:338)
      	at org.apache.fop.layoutmgr.AbstractBreaker.doLayout(AbstractBreaker.java:458)
      	at org.apache.fop.layoutmgr.PageBreaker.doLayout(PageBreaker.java:112)
      	at org.apache.fop.layoutmgr.PageSequenceLayoutManager.activateLayout(PageSequenceLayoutManager.java:143)
      	at org.apache.fop.area.AreaTreeHandler.endPageSequence(AreaTreeHandler.java:267)
      	at org.apache.fop.fo.pagination.PageSequence.endOfNode(PageSequence.java:130)
      	at org.apache.fop.fo.FOTreeBuilder$MainFOHandler.endElement(FOTreeBuilder.java:360)
      	at org.apache.fop.fo.FOTreeBuilder.endElement(FOTreeBuilder.java:190)
      	at org.apache.xalan.transformer.TransformerIdentityImpl.endElement(TransformerIdentityImpl.java:1102)
      	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
      	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
      	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
      	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
      	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
      	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
      	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
      	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
      	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
      	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
      	at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:485)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

      Cause:
      CachedRenderPagesModel.java creates URIs using following code:

      Code Snippet 2:

       String fname = "fop-page-" + page.getPageIndex() + ".ser";
       URI tempURI = tempBaseURI.resolve(fname);
      

      CachedRenderPagesModel.java uses createTempFile method inside ResourceResolverFactory.java to create serialized PageViewport objects in the tmp directory.

      Code Snippet 3:

       private File createTempFile(String path) throws IOException {
                  File tempFile = File.createTempFile(path, ".fop.tmp");
                  File oldFile = (File)this.tempFiles.put(path, tempFile);
                  if (oldFile != null) {
                      String errorMsg = oldFile.getAbsolutePath() + " has been already created for " + path;
                      boolean newTempDeleted = tempFile.delete();
                      if (!newTempDeleted) {
                          errorMsg = errorMsg + ". " + tempFile.getAbsolutePath() + " was not deleted.";
                      }
      
                      throw new IOException(errorMsg);
                  } else {
                      return tempFile;
                  }
              }
      

      Here the code is using a concurrent HashMap to map the URI path created in Code Snippet 2 with the temp file created and if it get replaced while storing, it throws an exception.

      In multi threaded environment, different threads can create same path in code snippet 2 and will always replace the entry created in map in code snippet 3, and will throw the exception.

      Solution:

      There are multiple ways we can make the URI (being used as key in the map) unique.

      1. We can use SecureRandom to generateĀ  a radom number and append it to the URI.
      2. Append thread name to the URI
      3. Use combination of above two solutions i.e., append both in the URI

      This will make sure that we get an unique path as key for the map across multiple threads and avoid this exception.
      The patch uses a SecureRandom to make this path unique across different threads in code snippet 2.

       String fname = "fop-page-" + page.getPageIndex() + "-" + sr.nextLong() + ".ser";
       URI tempURI = tempBaseURI.resolve(fname);
      

      Here sr is secure random object.

        Attachments

        1. cached-model.patch
          1 kB
          Piyush Khandelwal

          Activity

            People

            • Assignee:
              ssteiner Simon Steiner
              Reporter:
              piyush.khandelwal Piyush Khandelwal
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 48h
                48h
                Remaining:
                Remaining Estimate - 48h
                48h
                Logged:
                Time Spent - Not Specified
                Not Specified