Bug 53081 - WebappClassLoader causes java.lang.OutOfMemoryError in findResourceInternal()
Summary: WebappClassLoader causes java.lang.OutOfMemoryError in findResourceInternal()
Status: RESOLVED FIXED
Alias: None
Product: Tomcat 7
Classification: Unclassified
Component: Catalina (show other bugs)
Version: 7.0.26
Hardware: PC All
: P2 normal (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-04-13 21:46 UTC by Dmitry
Modified: 2014-02-17 13:42 UTC (History)
1 user (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dmitry 2012-04-13 21:46:53 UTC
When examining the code of WebappClassLoader#findResourceInternal() (http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.26/org/apache/catalina/loader/WebappClassLoader.java#3098) I came to the conclusion that findResourceInternal() always loads the binary content of the resource, however binary content is only used in e.g. findClassInternal() and obviously not needed in findResource(). In certain cases this can cause OutOfMemoryError, for example when a big media file is packaged with application.

The example of stack trace is given here:
http://stackoverflow.com/questions/10100480

Solution: the binary content should be loaded for certain types of resources (.classpath or .properties [only if fileNeedConvert is true]).
Comment 1 Christopher Schultz 2012-04-20 15:44:19 UTC
Stack trace is:

java.lang.OutOfMemoryError: Java heap space
    at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:3098)
    at org.apache.catalina.loader.WebappClassLoader.findResource(WebappClassLoader.java:1244)
    at org.apache.catalina.loader.WebappClassLoader.getResource(WebappClassLoader.java:1407)
    at org.springframework.core.io.ClassPathResource.exists(ClassPathResource.java:139)
    at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:248)
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:134)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at $Proxy30.open(Unknown Source)
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:93)
    at org.springframework.batch.core.step.item.ChunkMonitor.open(ChunkMonitor.java:105)
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:93)
    at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:301)
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:192)
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:135)
    at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:61)
    at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144)
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124)
    at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:281)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:120)
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)    
Line of code in 7.0.x/trunk and trunk is actually 3151:

  byte[] binaryContent = new byte[contentLength];

I would argue that instead of only loading certain types of resources (using what? magic number or other content-type-sniffing?), WebAppClassLoader could be configured never to load complete content for resources over a certain size.

I'd be interested to hear why resources are loaded in their entirety in this way. Another flavor of findResourceInternal(File,String) only creates URI objects and does not access the disk at all.
Comment 2 Mark Thomas 2012-05-29 13:15:44 UTC
As far as I can tell, the full resource has always been cached going back to the original implementation of the WebappClassLoader back in Tomcat 4.

This has been fixed in trunk and 7.0.x so the only things cached are a) classes (only temporarily until they are defined) and b) properties files needing encoding conversion. Note that the class loading code has been fragile in the past and it is possible that the fix may introduce a regression. If this is the case, it is likely the fix will be reverted and the advice will then be "don't load large static resources this way".

Assuming no problems emerge during testing, the fix will be in 7.0.28 onwards.
Comment 3 vladk 2013-11-06 13:44:33 UTC
We are observing significant performance degradation caused by this fix. In our case, the DocumentBuilderFactory.newInstance() method looks for class path resources. Before that fix the WebAppClassloader cached resources it had found and only the first call was slow. Now it scans jars on the classpath in each call and reading a big signed jar takes quite long.

We should be able to solve our problem by using javax.xml.parsers.DocumentBuilderFactory system property, but this change in the WebAppClassloader can cause performance degradation at other places.

Is there any better way to solve the mentioned OutOfMemoryError?
Comment 4 Christopher Schultz 2013-11-06 16:17:00 UTC
I think you should open a new bug report and reference this old one in that new report. Your report will get more visibility that way.