[SYSTEMDS-1759] make UDFs callable from expressions

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: SystemML 1.0.0
• Fix Version/s: SystemML 1.1
• Component/s: Parser
• Environment: IBM DataScience Experience
• Assignee: Matthias Boehm
• Reporter: Romeo Kienzer
• Created: 11/Jul/17 20:44
• Updated: 05/Mar/18 06:55
• Resolved: 05/Mar/18 06:55

Description

SystemML Parser stops with exeption:

only builtin functions allowed as part of
expression

But the following re-write could be automated during parsing

delta3 = -(Y-yHat) * sigmoidPrime(z3)
=>
smp = sigmoidPrime(z3)
delta3 = -(Y-yHat) * smp

Please consider the following code:

script = """

#
sigmoid = function(matrix[double] z) return (matrix[double] z)

{ z = 1/(1+exp(-z)) }

sigmoidPrime = function(matrix[double] z) return (matrix[double] z)

{ #Gradient of sigmoid z = exp(-z)/(1+exp(-z)) }

X=matrix("3 5 5 1 10 2", rows=3, cols=2)
inputLayerSize = 2
outputLayerSize = 1
hiddenLayerSize = 3

W1 = rand(rows=inputLayerSize,cols=hiddenLayerSize)
W2 = rand(rows=hiddenLayerSize,cols=outputLayerSize)

feedForward = function (matrix[double] X,
matrix[double] W1,
matrix[double] W2) return (matrix[double] z2,matrix[double] z3,matrix[double] Y)

{ z2 = X %*% W1 a2 = sigmoid(z2) z3 = (a2 %*% W2) Y = sigmoid(z3) }

gradient = function(matrix[double] X,
matrix[double] W1,
matrix[double] W2,
matrix[double] Y) return (matrix[double] dJdW1,matrix[double] dJdW1)

{ #Compute derivative with respect to W and W2 for a given X and y: [z2,z3,Yhat] = feedForward(X,W1,W2) delta3 = -(Y-yHat) * sigmoidPrime(z3) dJdW2 = t(a2) %*% delta3 delta2 = (delta3 %*% t(W2))*sigmoidPrime(z2) dJdW1 = t(X) %*% delta2 }

[z2,z3,Yhat]=feedForward(X,W1,W2)
nrx = nrow(X)
ncx = ncol(X)
nrw1 = nrow(W1)
ncw1 = ncol(W1)
"""

I'm getting a parser exeption saying that

Caused by: org.apache.sysml.parser.ParseException:
-------------------------------------------------------------- The following 2 parse issues were encountered:
1 [line 39:25] [Validation error] -> delta3 = -(Y-yHat) * sigmoidPrime(z3) only builtin functions allowed as part of
expression
2 [line 42:32] [Validation error] -> delta2 = (delta3 %*% > t(W2))*sigmoidPrime(z2) only builtin functions allowed as part of
expression

Issue Links

• duplicates: SYSTEMDS-1444 UDFs w/ single output in expressions