Related to 51919. Currently for each download of resources, HttpSamplerBase will create a pool of Threads, do download and shutdown pool. This results in lots of costly creations/destruction of Threads. It would be better to create Pool once for Thread Group and make Samplers reuse it (as proposed by sebb)
Created attachment 31959 [details] BROKEN PATCH to illustrate dev list discussion
PR https://github.com/apache/jmeter/pull/132 Please test and review carefully A shared thread pool is used for all the parallel downloads. A throttle mechanism is used per user to ensure the max concurrent downloads setting is not exceeded. A new parameter httpsampler.parallel_download_thread_keepalive_inseconds is used to set the thread TTL when idle (default 60s). the parameter httpsampler.await_termination_timeout is not used anymore and has been removed. TODO : ResourcesDownloader#shrink should be called at the end of a test to accelerate the release of the thread pool idle threads (not mandatory)
I've closed the PR as it's hard to have both Bug 59034 and this one. I will retry after the jmeter 3.0 release
Created attachment 33580 [details] patch attach the pr 132 as patch
PR https://github.com/apache/jmeter/pull/155 Back dans les bacs à chaque attaque, contact ! new version of the PR which should play nicely with Bug 59034. the cache manager is now correctly shared between the threads of a given user. Description : Embedded Resources concurrent download : Improve performances by avoiding shutdown of ThreadPoolExecutor at each sample. A shared thread pool is used for all the parallel downloads. A throttle mechanism is used per user to ensure the max concurrent downloads setting is not exceeded. A new parameter httpsampler.parallel_download_thread_keepalive_inseconds is used to set the thread TTL when idle (default 60s). the parameter httpsampler.await_termination_timeout is not used anymore and has been removed. TODO : ResourcesDownloader#shrink should be called at the end of a test to accelerate the release of the thread pool idle threads (not mandatory) the httpclient is shared between the threads of a given virtual user. the cache is shared between the threads of a given virtual user.
Created attachment 33651 [details] Benchmark test plan
Hello, I benchmarked the PR 155 by Benoit W.(that he updated this evening), the gain of Throughtput with 15 threads is 1.65 bigger than the throughput without it. I will be commiting it tomorrow, unless there is opposition. Regards Benchmark conditions: - Tomcat with session timeout 1 minute - Keepalive on connector: 2s <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" enableLookups="false" keepAliveTimeout="2000" minSpareThreads="50" acceptorThreadCount="2" compression="off" redirectPort="8443" /> - Log level to WARN JMeter in Non GUI mode: -server -Xms1024m -Xmx1024m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xss256k -Dcom.sun.management.jmxremote.port=3333 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime -XX:+PrintTenuringDistribution -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCCause -Xloggc:/data/jmeter/verbose-3.0.log -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false
Author: pmouawad Date: Wed Mar 9 12:23:31 2016 New Revision: 1734228 URL: http://svn.apache.org/viewvc?rev=1734228&view=rev Log: [Bug 52073] Embedded Resources Parallel download : Improve performances by avoiding shutdown of ThreadPoolExecutor at each sample Based on PR by Benoit Wiart + the addition (blame me) of JMeterPoolingClientConnectionManager (see mailing list mail I will send) Bugzilla Id: 52073 Added: jmeter/trunk/src/protocol/http/org/apache/http/ jmeter/trunk/src/protocol/http/org/apache/http/impl/ jmeter/trunk/src/protocol/http/org/apache/http/impl/conn/ jmeter/trunk/src/protocol/http/org/apache/http/impl/conn/JMeterPoolingClientConnectionManager.java (with props) jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/ResourcesDownloader.java (with props) Modified: jmeter/trunk/bin/jmeter.properties jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/control/CacheManager.java jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPHC4Impl.java jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSamplerBase.java jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/MeasuringConnectionManager.java jmeter/trunk/xdocs/changes.xml
Date: Fri Mar 18 09:25:58 2016 New Revision: 1735560 URL: http://svn.apache.org/viewvc?rev=1735560&view=rev Log: Followup to r17734228. No need for temporary variable. Return CacheManager directly. Bugzilla Id: 52073 Modified: jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/control/CacheManager.java
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/2617