[SYSTEMDS-3702] Using eval() with rand() in remote parfor throws error

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: SystemDS 3.2
• Fix Version/s: SystemDS 3.3
• Component/s: Compiler, ParFor, Spark
• Environment: systemds-3.2.0 ; ; Spark version 3.3.1 ; Using Scala version 2.12.15, OpenJDK 64-Bit Server VM, 11.0.22 ; Branch HEAD ; Compiled by user yumwang on 2022-10-15T09:47:01Z ; Revision fbbcf9434ac070dd4ced4fb9efe32899c6db12a9
• Assignee: Matthias Boehm
• Reporter: Louis Le Page
• Created: 28/May/24 15:22
• Updated: 02/Jun/24 09:41
• Resolved: 02/Jun/24 09:41

Description

TLDR: Some functions called using eval() in a remote parfor are compiled into SP instructions, but only CP instructions are expected.

Problem:

When using an eval() call in a remote parfor in spark mode, it seems like it is compiled into SP instructions, but only CP instructions are expected.

This only happens when using eval() in a parfor-loop that is distrubuted on cluster cluster, as described in the example code.
If eval() is running a function that calls matrix() or rand(), it is compiled into a org.apache.sysds.runtime.instructions.spark.RandSPInstruction, which is not expected:

PARFOR: Failed to execute loop in parallel.
...
Caused by: org.apache.sysds.runtime.DMLRuntimeException: 
Not supported: Instructions of type other than CP instructions org.apache.sysds.runtime.instructions.spark.RandSPInstruction

Code Example:

myFunction(100)

myFunction = function(Integer X)
return(Matrix[Double] A){
  model_function="dummy_rand_model"
  Y = 200  
  A = matrix(0, rows=X, cols=Y)

  #use below parfor to force in spark
  parfor (i in 1:X, opt=CONSTRAINED, mode=REMOTE_SPARK ){
  #for (i in 1:X){    
    #use function directly
    #P = rand(rows=1, cols=Y )    
    
    #use eval
    P = eval(model_function, list(Y=Y))    
    A[i] = P
  }  
  print("Avg:\n"+toString(avg(A)))
}

dummy_rand_model = function(Integer Y)
return( Matrix[Double] P){
        P = matrix(1, rows=1, cols=Y)
}

It does work correctly when directly using the rand() function directly without eval().

From my testing, I found that for the error to occur, it is necessary to call parfor in a function and make the parfor dependent on a parameter of that function, in this example: X

Errors:

Attached is an -explain  output and a full stacktrace of the error.