Bug 48889 - Wrong response time with mode=Statistical and num_sample_threshold > 1
Wrong response time with mode=Statistical and num_sample_threshold > 1
Status: RESOLVED FIXED
Product: JMeter
Classification: Unclassified
Component: Main
2.3.4
PC All
: P2 major (vote)
: ---
Assigned To: JMeter issues mailing list
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2010-03-11 09:49 UTC by JottKa
Modified: 2010-03-11 23:40 UTC (History)
0 users



Attachments
proposed patch (856 bytes, patch)
2010-03-11 12:57 UTC, JottKa
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description JottKa 2010-03-11 09:49:36 UTC
I’ve got a master-slave setup and need to do statistical sampling because otherwise the master dies on OutOfMemory. I use an Aggregate Graph/Report as recommended. The calculated values for hits/sec reported by the listeners are wrong.

To analyze the situation use I the following testplan:

-	Threadgroup with 10 threads running for 100 iterations
-	BeanShell Sampler sleeping for 500 millis
-	AggregateGraph/Report


Now the results must be more or like:
-	Hits/sec just below 20 (like 19.6)
-	Average response time approx 500 millis

Local execution delivers this result.

Set num_sample_threshold=1 and time_threshold=3600 delivers this result.

Set num_sample_threshold=100 time_threshold=3600 delivers
-	Hits/sec 19.7
-	Average response time 56!

The response time varies with the sample_threshold but always is way to low.
The higher the sample_threshold the higher the delta between real and reported response time
Comment 1 JottKa 2010-03-11 12:57:40 UTC
Created attachment 25115 [details]
proposed patch
Comment 2 Sebb 2010-03-11 14:58:30 UTC
Thanks for the suggested patch, which should help.

However, I think the threadGroup is still needed, otherwise unrelated threads in different thread groups will be aggregated, leading to the same problem.
Comment 3 JottKa 2010-03-11 15:55:35 UTC
(In reply to comment #2)
> Thanks for the suggested patch, which should help.
> 
> However, I think the threadGroup is still needed, otherwise unrelated threads
> in different thread groups will be aggregated, leading to the same problem.

Yes but at least with the current implementation the event.getResult().getThreadName() call already contains the threadGroup name. A typical key would be "NameOfSampler-NameOfThreadGroup 1-10". I just verified the proper working with two threadgroups. But an explicit call to getThreadGroup would make it more robust.
Comment 4 Sebb 2010-03-11 23:40:11 UTC
Turns out that the problem was not really due to aggregating across thread groups.

The StatisticalSampleResult now maintains the elapsed time directly, rather than trying to fake it by setting the start and end times.

See:

URL: http://svn.apache.org/viewvc?rev=922072&view=rev
Log:
Bug 48889 - Wrong response time with mode=Statistical and num_sample_threshold > 1
See also SVN revisions: 922069,922067,922055,922054,922051