Bug 54990 - Download large files avoiding outOfMemory
Download large files avoiding outOfMemory
Status: RESOLVED FIXED
Product: JMeter
Classification: Unclassified
Component: HTTP
2.9
PC Windows XP
: P2 enhancement (vote)
: ---
Assigned To: JMeter issues mailing list
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2013-05-18 14:01 UTC by Roberto Contiero
Modified: 2013-06-03 19:41 UTC (History)
1 user (show)



Attachments
The Jmeter environment (37.75 KB, image/jpeg)
2013-05-19 13:40 UTC, Roberto Contiero
Details
Conditions before the read of the stream (443.38 KB, image/jpeg)
2013-05-19 13:41 UTC, Roberto Contiero
Details
Conditions after the read of the stream (435.48 KB, image/jpeg)
2013-05-19 13:43 UTC, Roberto Contiero
Details
Conditions before returning from the readResponse method (423.33 KB, image/jpeg)
2013-05-19 13:45 UTC, Roberto Contiero
Details
Conditions after returning from the readResponse method (454.50 KB, image/jpeg)
2013-05-19 13:47 UTC, Roberto Contiero
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Roberto Contiero 2013-05-18 14:01:59 UTC
During tests of my web application, I need to download large file and save it on disk. To accomplish this I use an http request with a POST method to request the file and a "Save Response to a file" listener to save on disk the file. The size of the file to download is 441MB. My Jmeter (executed inside Windows XP 32 bit) started with -Xms1536m -Xmx1536m but, however, it goes to an OutOfMemory when start to write the file.
Debugging the source I notice that the OutOfMemory succeed in the class ResultSaver.java, specifically at this line

   pw.write(s.getResponseData());

of the method:

   private void saveSample(SampleResult s, int num)

When java call s.getResponseData(), it get an array of byte of the size 441MB. The write method of the FileOutputStream class allocates the same space for writing data to a file, causing OutOfMemory.
I have resolved this problem deleting the listener "Save Response to a file" and adding "BeanShell PostProcessor" with this code:

   final int BUFFER_SIZE = 65536;
   FileOutputStream fos = new  FileOutputStream("c:\\temp\\roberto\\ranemetestdwn.zip");
   for(int i = 0; i < data.length; i += BUFFER_SIZE) {
       fos.write(data, i, Math.min(data.length-i, BUFFER_SIZE));
   }
   fos.close();

Writing the file recursively, with a buffer of 64KB, none OutOfMemory error occur.
Then, if no other side effects exist, I propose to use similar code in class ResultSaver.java, in place of 

   pw.write(s.getResponseData());
Comment 1 Sebb 2013-05-18 14:45:19 UTC
(In reply to comment #0)
> When java call s.getResponseData(), it get an array of byte of the size
> 441MB. 

s.getResponseData() returns a pointer to the data array.

> The write method of the FileOutputStream class allocates the same
> space for writing data to a file, causing OutOfMemory.

Are you sure about that?
Which version of Java are you using?
Do you have a stack trace?

If FOS really does behave that way, the propsed fix would obviously help reduce memory requirements.
Comment 2 Roberto Contiero 2013-05-19 13:40:19 UTC
Created attachment 30299 [details]
The Jmeter environment

In this screeshot you can see the JVM used during test
Comment 3 Roberto Contiero 2013-05-19 13:41:31 UTC
Created attachment 30300 [details]
Conditions before the read of the stream
Comment 4 Roberto Contiero 2013-05-19 13:43:07 UTC
Created attachment 30301 [details]
Conditions after the read of the stream
Comment 5 Roberto Contiero 2013-05-19 13:45:44 UTC
Created attachment 30302 [details]
Conditions before returning from the readResponse method
Comment 6 Roberto Contiero 2013-05-19 13:47:26 UTC
Created attachment 30303 [details]
Conditions after returning from the readResponse method
Comment 7 Roberto Contiero 2013-05-19 13:52:50 UTC
Sorry, I didn't explain myself properly. It's obvious to me that s.getResponseData() returns a pointer to the data array, but returning a pointer to entire array, pw.write() allocates that space.
I have reproduced the issue and I have attached some screenshots that explain my test. The jvm.jpg is my jmeter environment. The beforeReadStream.jpg shows the memory status before read the stream (on the right of the image you can see a Java VisualVM screenshot). When the stream has been readed, in the afterReadStream.jpg you can see that allocated memory is about 500MB. The w.toByteArray() instruction (see beforeReturnFromReadStream.jpg), at the end of the readResponse method of the class HTTPSamplerBase, requires as much memory (see afterReturnFromReadStream.jpg). Then, when I reach the pw.write(s.getResponseData()); line and execute it, I obtain an OutOfMemoryError (below, the stack trace from the beginning of the test):


2013/05/19 10:53:45 INFO  - jmeter.engine.StandardJMeterEngine: Listeners will be started after enabling running version 
2013/05/19 10:53:45 INFO  - jmeter.engine.StandardJMeterEngine: To revert to the earlier behaviour, define jmeterengine.startlistenerslater=false 
2013/05/19 10:53:45 INFO  - jmeter.engine.StandardJMeterEngine: Running the test! 
2013/05/19 10:53:45 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(true,*local*) 
2013/05/19 10:53:45 INFO  - jmeter.engine.StandardJMeterEngine: Starting ThreadGroup: 1 : simone 
2013/05/19 10:53:45 INFO  - jmeter.engine.StandardJMeterEngine: Starting 1 threads for group simone. 
2013/05/19 10:53:45 INFO  - jmeter.engine.StandardJMeterEngine: Thread will stop on error 
2013/05/19 10:53:45 INFO  - jmeter.threads.ThreadGroup: Starting thread group number 1 threads 1 ramp-up 60 perThread 60000.0 delayedStart=false 
2013/05/19 10:53:45 INFO  - jmeter.threads.JMeterThread: jmeterthread.startearlier=true (see jmeter.properties) 
2013/05/19 10:53:45 INFO  - jmeter.threads.JMeterThread: Running PostProcessors in forward order 
2013/05/19 10:53:45 INFO  - jmeter.threads.ThreadGroup: Started thread group number 1 
2013/05/19 10:53:45 INFO  - jmeter.engine.StandardJMeterEngine: All thread groups have been started 
2013/05/19 10:53:45 INFO  - jmeter.threads.JMeterThread: Thread started: simone 1-1 
2013/05/19 10:53:45 INFO  - jmeter.protocol.http.sampler.HTTPHCAbstractImpl: Local host = dell 
2013/05/19 10:53:45 INFO  - jmeter.protocol.http.sampler.HTTPHC3Impl: HTTP request retry count = 1 
2013/05/19 10:53:45 INFO  - jmeter.util.JsseSSLManager: Using default SSL protocol: SSLv3 
2013/05/19 10:53:45 INFO  - jmeter.util.JsseSSLManager: SSL session context: per-thread 
2013/05/19 10:53:45 INFO  - jmeter.util.SSLManager: JmeterKeyStore Location:  type JKS 
2013/05/19 10:53:45 INFO  - jmeter.util.SSLManager: KeyStore created OK 
2013/05/19 10:53:45 WARN  - jmeter.util.SSLManager: Keystore file not found, loading empty keystore 
2013/05/19 11:03:30 ERROR - jmeter.threads.JMeterThread: Test failed! java.lang.OutOfMemoryError
	at java.io.FileOutputStream.writeBytes(Native Method)
	at java.io.FileOutputStream.write(Unknown Source)
	at org.apache.jmeter.reporters.ResultSaver.saveSample(ResultSaver.java:189)
	at org.apache.jmeter.reporters.ResultSaver.processSample(ResultSaver.java:150)
	at org.apache.jmeter.reporters.ResultSaver.sampleOccurred(ResultSaver.java:140)
	at org.apache.jmeter.threads.ListenerNotifier.notifyListeners(ListenerNotifier.java:84)
	at org.apache.jmeter.threads.JMeterThread.notifyListeners(JMeterThread.java:783)
	at org.apache.jmeter.threads.JMeterThread.process_sampler(JMeterThread.java:442)
	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256)
	at java.lang.Thread.run(Unknown Source)

2013/05/19 11:03:30 INFO  - jmeter.threads.JMeterThread: Thread finished: simone 1-1 
2013/05/19 11:03:30 INFO  - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test 
2013/05/19 11:03:30 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(false,*local*)
Comment 8 Sebb 2013-05-19 14:31:22 UTC
Thanks for the stacktrace etc.

Looks like the memory allocation must be in the native method, as that is what causes the OOME in the stacktrace.

So I did some research and it seems that native code which accesses a Java array may involve copying the array (depending on the JVM):

http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/design.html#wp1265

This is obviously a general issue with passing large arrays to native methods, and presumably affects many other file operations as they are likely to end up calling native methods.

Need to check whether there are other parts of JMeter that can write (or possibly read) large arrays.
Comment 9 Roberto Contiero 2013-05-19 15:32:56 UTC
Thank you for the tip, Sebastian. It has been my pleasure to help you.
By
Comment 10 Sebb 2013-05-19 23:26:23 UTC
URL: http://svn.apache.org/r1484366
Log:
Download large files avoiding outOfMemory
Add utility method to do chunking and use in ResultSaver
Bugzilla Id: 54990

Modified:
    jmeter/trunk/src/core/org/apache/jmeter/reporters/ResultSaver.java
    jmeter/trunk/src/jorphan/org/apache/jorphan/util/JOrphanUtils.java
    jmeter/trunk/xdocs/changes.xml
Comment 11 Philippe Mouawad 2013-05-29 20:09:24 UTC
Date: Wed May 29 20:07:20 2013
New Revision: 1487627

URL: http://svn.apache.org/r1487627
Log:
Bug 54990 - Download large files avoiding outOfMemory
Use method from commons-io
Bugzilla Id: 54990

Modified:
    jmeter/trunk/src/jorphan/org/apache/jorphan/util/JOrphanUtils.java
Comment 12 Philippe Mouawad 2013-06-03 19:41:32 UTC
Date: Mon Jun  3 19:40:17 2013
New Revision: 1489124

URL: http://svn.apache.org/r1489124
Log:
Bug 54990 - Download large files avoiding outOfMemory
Rollback as per sebb comment on dev list
Bugzilla Id: 54990

Modified:
    jmeter/trunk/src/jorphan/org/apache/jorphan/util/JOrphanUtils.java