Bug 40369 - LDAP: Stable search results in sampler + added "Equals" assertion
LDAP: Stable search results in sampler + added "Equals" assertion
Status: RESOLVED FIXED
Product: JMeter
Classification: Unclassified
Component: Main
2.2
All other
: P2 enhancement (vote)
: ---
Assigned To: JMeter issues mailing list
: PatchAvailable
Depends on: 40381
Blocks:
  Show dependency tree
 
Reported: 2006-08-30 23:55 UTC by nrhope
Modified: 2007-04-22 15:42 UTC (History)
0 users



Attachments
use next patch instead (replaced a > with a >=) (25.69 KB, patch)
2006-08-30 23:57 UTC, nrhope
Details | Diff
use this patch instead (25.69 KB, patch)
2006-09-01 13:05 UTC, nrhope
Details | Diff
improved diff format, included patch for 40381 (25.69 KB, patch)
2006-09-05 06:16 UTC, nrhope
Details | Diff
improved diff format, included patch for 40381 (33.14 KB, patch)
2006-09-05 06:31 UTC, nrhope
Details | Diff
improve config of diff format + map to/from symbolic descriptions (36.23 KB, patch)
2006-09-06 07:00 UTC, nrhope
Details | Diff
remove fix for "asserion.test_string" spelling error (37.42 KB, patch)
2006-10-03 21:49 UTC, nrhope
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description nrhope 2006-08-30 23:55:50 UTC
Also fixes #40103.

I have made the following improvements to the JMeter LDAP support
which I wanted to share with the list:
   1. Changed LDAP sampler to:
       a. sort search results (by DN), attributes for each result (by
id) and values (by toString()) for searches returning less the a
configured constant (1000) entries. Having a stable ordering allows a
new class of response assertion detailed below.
       b. fix bugzilla 40103 (extra "</operation>").
       c. use StringBuffer instead of '+' for search results where
there may be big numbers of results to be marshalled to XML.
       d. Added extra <searchresults> tag and nested <dn> ...
<attribute> stuff inside the <searchresult> tag. This was because the
existing structure is not very XSD/JAXB etc friendly, and there seems
no good reason why it shouldn't be.
   2. Changed ResponseAssertion.java to:
       a. support a new "Pattern Matching Rule" called "Equals" which
does a direct string comparison between the response and the first
"Patterns To Test" string which has been entered. This is not directly
tied to the LDAP sampler but requires a sampler to output fields in a
stable order to be used effectively. It is better for simple
comparing against a full response then "Matches" because there is no
need to consider quoting of special regex chars.
       b. log.info() information about failed assertions so that the
reason for failures is clear immediately from the log, including the
assertion's name. In the case of the new "Equals" assertions, a small
diff showing markers for initial matching sequence (if any) and final
matching sequence (if any) is also logged, so that the non-matching
text is aligned in the middle for easy identification.

These changes greatly improve the ease with which you can script an
LDAP interaction and then use the results seen in the View Results
Tree to turn the interaction into a unit test asserting correct
behaviour (after suitable manual review of course).
Comment 1 nrhope 2006-08-30 23:57:14 UTC
Created attachment 18772 [details]
use next patch instead (replaced a > with a >=)

Against rel-2.2 branch
Comment 2 nrhope 2006-09-01 13:05:02 UTC
Created attachment 18799 [details]
use this patch instead

changed a > to a >=
Comment 3 nrhope 2006-09-05 06:16:34 UTC
Created attachment 18820 [details]
improved diff format, included patch for 40381

Patch after thorough testing and changing the diff format to deal with longer
strings (now looks like

received   : initial...<<delta...>>final...
comparison : initial...<<delta...>>final...

where all sections are present and longer then the limit defined by
"assertion.equals_section_diff_len=" property in jmeter.properties, which
defaults to 50 chars if not specified).

Includes patch for 40381 as was too difficult for me to maintain a separate
working directory.
Comment 4 nrhope 2006-09-05 06:31:05 UTC
Created attachment 18821 [details]
improved diff format, included patch for 40381

Previous patch was obsolete patch from wrong working directory (bugzilla was
reporting wrong size).
Comment 5 nrhope 2006-09-06 07:00:23 UTC
Created attachment 18824 [details]
improve config of diff format + map to/from symbolic descriptions

Still contains patch for 40381. Now seq at start and end of diff delta can be
configured via jmeter.properties ("[[[" and "]]]" by default) and default diff
delta section length increased to 100 chars after observing real-world diff
output). Added methods to map scope and modification ops to/from strings for
easier reuse/better clarity.
Comment 6 nrhope 2006-09-06 08:15:19 UTC
Also note that last patch fixed a typo in ResponseAssertion.java where

   public final static String TEST_STRINGS = "Asserion.test_strings"; 

should be

   public final static String TEST_STRINGS = "Assertion.test_strings"; 
Comment 7 Sebb 2006-09-17 23:03:43 UTC
(In reply to comment #6)
> Also note that last patch fixed a typo in ResponseAssertion.java where
> 
>    public final static String TEST_STRINGS = "Asserion.test_strings"; 
> 
> should be
> 
>    public final static String TEST_STRINGS = "Assertion.test_strings"; 
> 

Unfortunately, fixing that would invalidate any existing test plans, as the name
is stored in the plan. But it does not matter; the name is largely irrelevant.
Comment 8 Sebb 2006-09-17 23:37:29 UTC
The commented out main method generates test results of the form:

received  : 
received  : [[[aaa]]]
comparison: 
comparison: [[[zzz]]]

The first received and comparison lines are always blank - so what is their
purpose? Is there a bug in the equalsComparisonText() method?
Comment 9 nrhope 2006-09-20 04:03:29 UTC
(In reply to comment #7)
> Unfortunately, fixing that would invalidate any existing test plans, as the name
> is stored in the plan. But it does not matter; the name is largely irrelevant.
> 

True, but its just a global search and replace. Not a biggy if not included in
the applied patch, but I thought while it was alpha it might be a good idea...

Comment 10 nrhope 2006-09-20 04:09:05 UTC
(In reply to comment #8)
> The commented out main method generates test results of the form:
> 
> received  : 
> received  : [[[aaa]]]
> comparison: 
> comparison: [[[zzz]]]
> 
> The first received and comparison lines are always blank - so what is their
> purpose? Is there a bug in the equalsComparisonText() method?


These lines are just to provide better visibility for the [[[ text following, as
I found them hard to spot visually when there is lots of XML logged to
jmeter.log. Again, no biggy...
Comment 11 Sebb 2006-09-20 10:21:28 UTC
(In reply to comment #9)
> (In reply to comment #7)
> > Unfortunately, fixing that would invalidate any existing test plans, as 
the name
> > is stored in the plan. But it does not matter; the name is largely 
irrelevant.
> > 
> True, but its just a global search and replace. Not a biggy if not included 
in
> the applied patch, but I thought while it was alpha it might be a good 
idea...

The proposed correction relates to a string that is in *released* code, which 
would mean global search and replace in every single test plan using a 
response assertion. This could be a very large number.
Comment 12 nrhope 2006-09-21 07:07:49 UTC
(In reply to comment #11)
> (In reply to comment #9)
> > (In reply to comment #7)
> > > Unfortunately, fixing that would invalidate any existing test plans, as 
> the name
> > > is stored in the plan. But it does not matter; the name is largely 
> irrelevant.
> > > 
> > True, but its just a global search and replace. Not a biggy if not included 
> in
> > the applied patch, but I thought while it was alpha it might be a good 
> idea...
> 
> The proposed correction relates to a string that is in *released* code, which 
> would mean global search and replace in every single test plan using a 
> response assertion. This could be a very large number.
> 
Right you are; I was forgetting this was in the generic assertion code. My bad.
Comment 13 nrhope 2006-10-03 21:49:44 UTC
Created attachment 18960 [details]
remove fix for "asserion.test_string" spelling error

Also improved output from Equals so only outputs diff text rather then complete
string, making it much easier to see what the offending text is
Comment 14 Sebb 2007-02-10 04:03:28 UTC
I've added the Response Assertion "Equals" enhancement.

Had to fix a bug - the equals code only checked the first entry, which does 
not make sense for negative matching: one might want to check that something 
does not match several different strings.

[By the way, please don't change the spacing etc of existing code; it makes 
reviewing and applying patches much harder. I had to fix a lot of tabs/spaces 
to work out what was actually changed. Likewise changing imports to use .*]

The LDAP changes have not yet been reviewed or applied.
Comment 15 Sebb 2007-04-22 15:42:44 UTC
Fixed in SVN in r531295

Had to add some LimitException handling to the formatting routines.