[SYSTEMDS-590] Improve Namespace Handling for UDFs

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: SystemML 0.10
• Fix Version/s: SystemML 0.10
• Component/s: Parser
• Epic Link: Improve UDFs
• Assignee: Glenn Weidner
• Reporter: Mike Dusenberry
• Created: 22/Mar/16 19:08
• Updated: 12/May/16 18:42
• Resolved: 12/May/16 18:42

Description

Currently, if a UDF body involves calling another UDF, the default global namespace is assumed, unless a namespace is explicitly indicated. This becomes a problem when a file contains UDFs, and is then sourced from another script.

Imagine a file funcs.dml as follows:

f = function(double x, int a) return (double ans) {
  x2 = g(x)
  ans = a * x2
}

g = function(double x) return (double ans) {
  ans = x * x
}

Then, let's try to call f:

script = """
source ("funcs.dml") as funcs

ans = funcs::f(3, 1)
print(ans)
"""
ml.reset()
ml.executeScript(script)

This results in an error since f is in the funcs namespace, but the call to g assumes g is still in the default namespace. Clearly, the user intends to the use the g that is located in the same file.

Currently, we would need to adjust funcs.dml as follows to explicitly assume that f and g are in a funcs namespace:

f = function(double x, int a) return (double ans) {
  x2 = funcs::g(x)
  ans = a * x2
}

g = function(double x) return (double ans) {
  ans = x * x
}

Instead, it would be better to simply first look for g in its parent's namespace. In this case, the "parent" would be the function f, and the namespace we have selected is funcs, although that choice would be left up to the end-user. Then, namespace assumptions would not be necessary.

Issue Links

• is related to: SYSTEMDS-670 PyDML `random.normal` Function Broken Due to Namespace Issue
• relates to: SYSTEMDS-587 Improvements Triggered By Deep Learning Work

Sub-Tasks

• Override default namespace for imported custom functions: Closed
• Improve handling of source statements in scripts that are also imported: Closed
• Improve handling of namespace conflicts: Closed
• Handle circular import references: Closed
• DML Functions Should Override Builtin Functions: Closed