[FOP-2874] [PATCH] FOP conserve memory policy fails in multi-threaded environment

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.2, 2.3
• Flags: Patch
• Assignee: Simon Steiner
• Reporter: Piyush Khandelwal
• Created: 18/Jul/19 10:09
• Updated: 23/Jul/19 10:41
• Resolved: 23/Jul/19 10:41

Description

When conserve memory policy is enabled 
Code Snippet 1:

foUserAgent.setConserveMemoryPolicy(true);

in multi-threaded environment, it throws following exception

SEVERE: Error while serializing page 154. Reason: java.io.IOException: \tmp\fop-page-153.ser5056538250865196860.fop.tmp has been already created for /fop-page-153.ser
java.io.IOException: \tmp\fop-page-153.ser5056538250865196860.fop.tmp has been already created for /fop-page-153.ser
	at org.apache.fop.apps.io.ResourceResolverFactory$DefaultTempResourceResolver.createTempFile(ResourceResolverFactory.java:187)
	at org.apache.fop.apps.io.ResourceResolverFactory$DefaultTempResourceResolver.getOutputStream(ResourceResolverFactory.java:199)
	at org.apache.fop.apps.io.ResourceResolverFactory$TempAwareResourceResolver.getOutputStream(ResourceResolverFactory.java:159)
	at org.apache.fop.apps.io.ResourceResolverFactory$DefaultResourceResolver.getOutputStream(ResourceResolverFactory.java:126)
	at org.apache.fop.apps.io.InternalResourceResolver.getOutputStream(InternalResourceResolver.java:103)
	at org.apache.fop.area.CachedRenderPagesModel.savePage(CachedRenderPagesModel.java:132)
	at org.apache.fop.area.CachedRenderPagesModel.checkPreparedPages(CachedRenderPagesModel.java:112)
	at org.apache.fop.area.RenderPagesModel.addPage(RenderPagesModel.java:146)
	at org.apache.fop.layoutmgr.AbstractPageSequenceLayoutManager.finishPage(AbstractPageSequenceLayoutManager.java:316)
	at org.apache.fop.layoutmgr.PageSequenceLayoutManager.finishPage(PageSequenceLayoutManager.java:243)
	at org.apache.fop.layoutmgr.AbstractPageSequenceLayoutManager.makeNewPage(AbstractPageSequenceLayoutManager.java:287)
	at org.apache.fop.layoutmgr.PageSequenceLayoutManager.makeNewPage(PageSequenceLayoutManager.java:192)
	at org.apache.fop.layoutmgr.PageBreaker.handleBreakTrait(PageBreaker.java:633)
	at org.apache.fop.layoutmgr.PageBreaker.startPart(PageBreaker.java:511)
	at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:659)
	at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:604)
	at org.apache.fop.layoutmgr.AbstractBreaker.addAreas(AbstractBreaker.java:599)
	at org.apache.fop.layoutmgr.PageBreaker.doPhase3(PageBreaker.java:338)
	at org.apache.fop.layoutmgr.AbstractBreaker.doLayout(AbstractBreaker.java:458)
	at org.apache.fop.layoutmgr.PageBreaker.doLayout(PageBreaker.java:112)
	at org.apache.fop.layoutmgr.PageSequenceLayoutManager.activateLayout(PageSequenceLayoutManager.java:143)
	at org.apache.fop.area.AreaTreeHandler.endPageSequence(AreaTreeHandler.java:267)
	at org.apache.fop.fo.pagination.PageSequence.endOfNode(PageSequence.java:130)
	at org.apache.fop.fo.FOTreeBuilder$MainFOHandler.endElement(FOTreeBuilder.java:360)
	at org.apache.fop.fo.FOTreeBuilder.endElement(FOTreeBuilder.java:190)
	at org.apache.xalan.transformer.TransformerIdentityImpl.endElement(TransformerIdentityImpl.java:1102)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
	at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:485)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Cause:
CachedRenderPagesModel.java creates URIs using following code:

Code Snippet 2:

 String fname = "fop-page-" + page.getPageIndex() + ".ser";
 URI tempURI = tempBaseURI.resolve(fname);

CachedRenderPagesModel.java uses createTempFile method inside ResourceResolverFactory.java to create serialized PageViewport objects in the tmp directory.

Code Snippet 3:

 private File createTempFile(String path) throws IOException {
            File tempFile = File.createTempFile(path, ".fop.tmp");
            File oldFile = (File)this.tempFiles.put(path, tempFile);
            if (oldFile != null) {
                String errorMsg = oldFile.getAbsolutePath() + " has been already created for " + path;
                boolean newTempDeleted = tempFile.delete();
                if (!newTempDeleted) {
                    errorMsg = errorMsg + ". " + tempFile.getAbsolutePath() + " was not deleted.";
                }

                throw new IOException(errorMsg);
            } else {
                return tempFile;
            }
        }

Here the code is using a concurrent HashMap to map the URI path created in Code Snippet 2 with the temp file created and if it get replaced while storing, it throws an exception.

In multi threaded environment, different threads can create same path in code snippet 2 and will always replace the entry created in map in code snippet 3, and will throw the exception.

Solution:

There are multiple ways we can make the URI (being used as key in the map) unique.

1. We can use SecureRandom to generate  a radom number and append it to the URI.
2. Append thread name to the URI
3. Use combination of above two solutions i.e., append both in the URI

This will make sure that we get an unique path as key for the map across multiple threads and avoid this exception.
The patch uses a SecureRandom to make this path unique across different threads in code snippet 2.

 String fname = "fop-page-" + page.getPageIndex() + "-" + sr.nextLong() + ".ser";
 URI tempURI = tempBaseURI.resolve(fname);

Here sr is secure random object.