Bug 50898 - IncludeController : NullPointerException loading script in non-GUI mode if Includers use same element name
IncludeController : NullPointerException loading script in non-GUI mode if In...
Status: RESOLVED FIXED
Product: JMeter
Classification: Unclassified
Component: Main
2.5.1
All All
: P3 normal (vote)
: ---
Assigned To: JMeter issues mailing list
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2011-03-09 04:53 UTC by Luciana Moreira
Modified: 2012-05-22 22:49 UTC (History)
2 users (show)



Attachments
Including Plan that reproduces the NullPointer in non gui mode (7.21 KB, application/xml)
2011-10-01 22:31 UTC, Philippe Mouawad
Details
Included plan (77.37 KB, application/xml)
2011-10-01 22:31 UTC, Philippe Mouawad
Details
Log file (5.71 KB, text/plain)
2011-10-01 22:37 UTC, Philippe Mouawad
Details
Fix to issue (1.06 KB, patch)
2011-10-02 07:01 UTC, Philippe Mouawad
Details | Diff
Includer plan (4.04 KB, application/xml)
2011-10-02 17:00 UTC, Philippe Mouawad
Details
Included in Use.Include.jmx (8.48 KB, application/xml)
2011-10-02 17:01 UTC, Philippe Mouawad
Details
Included in Use.Include.jmx (4.96 KB, application/xml)
2011-10-02 21:16 UTC, Philippe Mouawad
Details
Includer plan (4.04 KB, application/octet-stream)
2011-10-02 21:18 UTC, Philippe Mouawad
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Luciana Moreira 2011-03-09 04:53:28 UTC
I am facing a problem which appeared after my colleague updated one of our scripts. He saved the script with the jmeter revision 1077948

I am able to load and execute my script without any problems if I do it from the GUI without providing it in the command line. However if I pass the script in the command line I get the following exception:

2010/09/20 17:12:09 ERROR - jmeter.engine.StandardJMeterEngine: Uncaught exception:  java.lang.NullPointerException
	at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:989)
	at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:989)
	at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:989)
	at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:989)
	at org.apache.jorphan.collections.HashTree.traverse(HashTree.java:971)
	at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:385)
	at java.lang.Thread.run(Thread.java:619)
	
	The problem seems to be in Iterator<?> iter = list().iterator();
            while (iter.hasNext()) {
                Object item = iter.next();
                final HashTree treeItem = getTree(item);
                visitor.addNode(item, treeItem);
                treeItem.traverseInto(visitor);
            }
     
     I don't understand how this problem can even happen. What occurs is that treeItem is null. 
     Adding some expressions evaluation I got the following results:
     list().contains(item) => true
     containsKey(item) => false
     get(item) != null => false
     
     I noticed that in the script that no longer works some elements are saved differently than they used to be:
     This is how it used to work:
     <org.apache.jmeter.protocol.smtp.sampler.SmtpSampler guiclass="org.apache.jmeter.protocol.smtp.sampler.gui.SmtpSamplerGui" testclass="org.apache.jmeter.protocol.smtp.sampler.SmtpSampler" testname="SMTP 1" enabled="true">
     
     This is how it is now:
     <SmtpSampler guiclass="SmtpSamplerGui" testclass="SmtpSampler" testname="SMTP 1" enabled="true">
     
     However I do not know if this is the cause of the problem...
     
     Any light in this matter would be very appreciated.
Comment 1 Luciana Moreira 2011-03-09 05:39:54 UTC
After digging more into this problem I found out a work around for my script.

The problem seems to come from IncludeController. I had in my script 2 IncludeControllers They had the same name and included the same file. By changing the name of one of the IncludeControllers I am now able to also launch JMeter passing this script as parameter.

I guess this bug is related to the overridden methods equals() and hashCode from AbstractTestElement. Somehow that confused the HashMap.
Comment 2 Philippe Mouawad 2011-10-01 22:31:30 UTC
Created attachment 27664 [details]
Including Plan  that reproduces the NullPointer in non gui mode
Comment 3 Philippe Mouawad 2011-10-01 22:31:57 UTC
Created attachment 27665 [details]
Included plan
Comment 4 Philippe Mouawad 2011-10-01 22:37:38 UTC
Created attachment 27666 [details]
Log file
Comment 5 Philippe Mouawad 2011-10-01 23:24:41 UTC
I confirm there is an issue related to hashCode and equals.

to test this hypothesis I modified HashTree#traverseInto:
private void traverseInto(HashTreeTraverser visitor) {

        if (list().size() == 0) {
            visitor.processPath();
        } else {
            Iterator<?> iter = list().iterator();
            
            while (iter.hasNext()) {
                Object item = iter.next();
                final HashTree treeItem = getTree(item);
                visitor.addNode(item, treeItem);
                if(treeItem==null)
                {
                    List<IncludeController>  list = new ArrayList<IncludeController>();
                    Iterator<?> iter2 = list().iterator();
                    while (iter2.hasNext()) {
                        Object item2 = iter2.next();
                        if(item2 instanceof IncludeController)
                        {
                            list.add((IncludeController)item2);
                        }
                    }
                    if(list.size()==2)
                    {
                        IncludeController controller1 = list.get(0);
                        IncludeController controller2 = list.get(1);
                        System.out.println(controller1.equals(controller2));
                        System.out.println(controller1.hashCode());
                        System.out.println(controller2.hashCode());
                    }
                }
                treeItem.traverseInto(visitor);
            }
        }
        visitor.subtractNode();
    }



And it shows the following:
true => Objects are equals
831453930
1248040939

According to javadocs:
If two objects are equal according to the equals(Object) method, then calling the hashCode method on each of the two objects must produce the same integer result.
Comment 6 Philippe Mouawad 2011-10-02 07:01:01 UTC
Created attachment 27671 [details]
Fix to issue

Patch relies on another more efficient iteration mode using entrySet.
This method will not use hashCode so will not meet bug.

traverse(HashTreeTraverser visitor) may have same issue:
- Is there a reason why getTree is called twice ? I don't think so cause it would meen visitor.addNode() changes data Map which would provoke ConcurrentModificationException ? if answer is no, then same iteration mode could be used

There is still the root cause to investigate regarding hashCode bug.

Regards
Philippe
Comment 7 Sebb 2011-10-02 14:16:34 UTC
(In reply to comment #2)
> Created attachment 27664 [details]
> Including Plan  that reproduces the NullPointer in non gui mode

Does not appear to be the correct test plan; it does not use an include controller
Comment 8 Sebb 2011-10-02 14:18:56 UTC
(In reply to comment #3)
> Created attachment 27665 [details]
> Included plan

If this is the sample included plan, it is excessively complicated.
Please use smallest possible test plan, and ideally use Java Request instead of HTTP sampler (unless the test requires an HTTP sample).
Comment 9 Philippe Mouawad 2011-10-02 17:00:09 UTC
Created attachment 27673 [details]
Includer plan
Comment 10 Philippe Mouawad 2011-10-02 17:01:05 UTC
Created attachment 27674 [details]
Included in Use.Include.jmx

Hello Sebb,
As you requested, much simpler plan.
Regards
Philippe
Comment 11 Sebb 2011-10-02 20:40:49 UTC
(In reply to comment #10)
> Created attachment 27674 [details]
> Included in Use.Include.jmx
> 
> Hello Sebb,
> As you requested, much simpler plan.

Thanks, that's better. 

Not sure why the Thread Group with the module controller is present, as the top-level script still fails without it in non-GUI mode. Likewise the Transaction Controller seems to be unnecessary.

All that is needed is a Test Fragment with a single Java sampler.
Comment 12 Philippe Mouawad 2011-10-02 21:16:04 UTC
Created attachment 27675 [details]
Included in Use.Include.jmx

Test plan as you requested.
Sebb, about your request, I just took another plan I found in another IncludeController bug 51869 and tried to reproduce issue with it by changing what initial bug submitter described, once it reproduced it, I used it as is.
That's why plan was initially complex.

Regards
Philippe
Comment 13 Philippe Mouawad 2011-10-02 21:18:27 UTC
Created attachment 27676 [details]
Includer plan
Comment 14 Sebb 2011-10-03 08:02:14 UTC
The fix looks OK, however it causes some tests to fail.

In particular, "ant batchtest" detects differences in BatchTestLocal.csv.
These show that the test plan behaves very differently.
Comment 15 Philippe Mouawad 2011-10-03 09:53:55 UTC
Hello Sebb,
Very strange indeed, as strange as the fact that although key is here, getTree does not find its value.

I think we should fix issue related to hashCode and equals in AbstractTestElement.

Regards
Philippe
Comment 16 Sebb 2011-10-03 10:31:18 UTC
(In reply to comment #15)
> Hello Sebb,
> Very strange indeed, as strange as the fact that although key is here, getTree
> does not find its value.

I think that can happen if the key changes whilst in the hash.

> I think we should fix issue related to hashCode and equals in
> AbstractTestElement.

That should help, but as has been seen from the patch failure, the behaviour of the JMeter code is quite tricky in this area.
Comment 17 Sebb 2012-05-21 01:59:57 UTC
The problem is in the ListedHashTree#replace method.

This updates the order of elements using the code:

order.set(order.indexOf(currentKey), newKey);

The problem is that indexOf() uses equals() to scan the LinkedList to find the matching Object. The current implementation of equals for TestElelemts compares contents. So if there are two identical Include Controllers the wrong key may be found. [This problem does not affect the tree itself, because that is a hash tree, and test elements use the identity hashcode.]

I suspect this does not affect the GUI tests because the Test Elements will then have additional properties which relate to the GUI.

The bug can be fixed by using Object equality for equals; unit tests all still work apart from TestHTTPFileArgs.testRemoving() which assumes that HTTPFileArgs are equal if they have equal contents. That could be fixed separately.

I'm not entirely sure what it means for two test elements to be truly "equal".
At present, they violate the rule that equal objects must have equal hash codes.

However, changing the AbstractTestElement#equals() method is a very big change and may have unexpected consequences (apart from just HTTPFileArgs).

As a workround for this bug, the IncludeController can be fixed to use Object equals.
After further testing, the fix can hopefully be extended to all test elements.
Comment 18 Sebb 2012-05-21 02:12:21 UTC
Added workround:

URL: http://svn.apache.org/viewvc?rev=1340884&view=rev
Log:
Bug 50898 - IncludeController : NullPointerException loading script in non-GUI mode if Includers use same element name

Modified:
   jmeter/trunk/build.xml
   jmeter/trunk/src/components/org/apache/jmeter/control/IncludeController.java
   jmeter/trunk/xdocs/changes.xml
Comment 19 Philippe Mouawad 2012-05-21 05:51:04 UTC
Great you fixed it Sebb .
I close issue.
Comment 20 Sebb 2012-05-22 22:49:41 UTC
Reverted IncludeController change.

Now use == rather than equals() when finding the entry in the list.

URL: http://svn.apache.org/viewvc?rev=1341670&view=rev
Log:
Bug 50898 - IncludeController : NullPointerException loading script in non-GUI mode if Includers use same element name
Apply better fix that applies for all elements, not just IncludeController

Modified:
   jmeter/trunk/src/components/org/apache/jmeter/control/IncludeController.java
   jmeter/trunk/src/jorphan/org/apache/jorphan/collections/ListedHashTree.java