[SCXML-213] Fixing the datamodel handling - reimplement xpath processing, the Data() function and the <assign> action

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0
• Fix Version/s: 2.0
• Assignee: Ate Douma
• Reporter: Ate Douma
• Created: 26/Oct/14 23:29
• Updated: 15/Nov/14 03:44
• Resolved: 15/Nov/14 03:32

Description

The current datamodel handling, using JAXP based xpath expressions, the custom Data() function and the implementation of the <assign> action is very limited, incomplete, and not at all SCXML spec compliant.

JAXP based xpath expressions require a predefined datatype to be specified for the result, which is pretty much a blocker, as it cannot be known upfront by the SCXML engine what the intended datatype should be.

As as consequence the current XPathEvaluator always converts the result of an normal (eval) expression to type String.
And the result of a location expression must be a NodeSet of which only the first Node element is returned.
For example an XPath variable cannot be used as assignment location, which this must be supported by the SCXML specification.

The Builtin#data() method (as implementation of the Data() function) always needs an intermediate single Node result from the #dataNode location method,
after which the Node its String content is tried to be converted to a Double, or else a Long, or as fallback the String value itself is returned.
Returning a Node or even a NodeSet currently is not supported (directly), except by using the (private) #dataNode method to evaluate the 'expr' parameter for the <assign> element.
The SCXML spec however requires much more extensive expressions, including using a NodeSet as value AND as target.

The SCXML <assign> element as result currently can only deal with a too limited set of use-cases, not even close to what is required to be supported by the SCXML spec.

To fix this properly will require some extensive (breaking) changes...

First of all, I plan to replace the JAXP xpath expression handling with Commons JXPath.
After tested and experimented with it for quite some time JXPath turns out to fit very nicely with the SCXML requirements.
And the JXPath support for xpath expressions applied to 'plain' Java beans makes it extremely convenient to support and 'navigate' nested SCXML system variables like the _event.data,
without having to 'convert' this into native DOM structures. The current JAXP based xpath expressions cannot deal with this at all.

Commons JXPath also makes it possible to properly implement xpath location expressions, returning JXPath pointer(s) which can be used to detect AND set left hand expressions,
including xpath variables (and nested Java bean paths as well).

And JXPath also will allow redefining and reimplementing the Data() function for non-xpath expression languages in a more 'native' and straightforward way, with support for non-flattened/converted return type data.

Furthermore JXPath can use and map the SCXML Context variables, and thus also the datamodel entries directly.
The Data() function now can be properly implemented requiring only an xpath expression instead of also requiring a data node variable to operate against:
So, instead of Data(node1, 'expr') we then can use Data('expr') where the node node1 itself is directly accessed within the expression.

With these changes and enhancements the <assign> action then can be reimplemented to support all the specification requirements.
To support location expressions targeting datamodel nodes or xpath variables for non-xpath based languages will require some extra customizations though to be able to access the 'pointer' of a location target.
For this purpose probably we need a separate location expression attribute, e.g. xlocation="" instead of location="" to hint the SCXML engine the location needs to be evaluated as xpath expression returning pointers, not data.

All these changes will be rather invasive and potentially breaking for existing SCXML documents using the Data() function and <assign> element.
And likely this will take several steps to implement properly with a potentially unstable intermediate state of the SCXML engine implementation.

I'll create separate sub tasks for these steps: the JAXP replacement with Commons JXPath, the Data() function and related Builtin class changes, and the changes to the <assign> action.

Until all these sub tasks are completed the current SCXML trunk might thus become unstable and probably should not be used/tested for current projects...

Once everything is working again as intended and this umbrella issue has been resolved, I'll give a heads-up on the users@ list with comprehensive explanations and instructions how to 'migrate' to the new implementation.

Issue Links

• is related to: SCXML-94 <assign> does not support non-XML datamodels
• is related to: SCXML-59 <assign location="..." expr="..."> problem for context with different internal XML representation
• is related to: SCXML-70 Inconsistent behavior between Data(name, '.'), single-node XML tree and multiple-node XML tree within <send> namelist
• supercedes: SCXML-94 <assign> does not support non-XML datamodels
• supercedes: SCXML-154 processing of param children of send actions
• supercedes: SCXML-59 <assign location="..." expr="..."> problem for context with different internal XML representation