[SYSTEMDS-3418] Cleaning Pipelines: Replace function failure in hybrid execution - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: SystemDS 3.1
Fix Version/s: SystemDS 3.1
Component/s: None
Labels:
None

Description

org.apache.sysds.runtime.DMLRuntimeException: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 1 and 34 -- Error evaluating instruction: CP°fcall°.builtinNS°s_executePipeline°true°12°8°pipeline=_fVar2767·FRAME·FP64·false°Xtrain=_mVar2769·MATRIX·FP64°Ytrain=_mVar2770·MATRIX·FP64°Xtest=_mVar2772·MATRIX·FP64°Ytest=_mVar2773·MATRIX·FP64°metaList=_Var2780·LIST·UNKNOWN·false°hyperParameters=_mVar2782·MATRIX·FP64°flagsCount=5·SCALAR·INT64·true°test=true·SCALAR·BOOLEAN·true°verbose=false·SCALAR·BOOLEAN·true°hpForPruning=_mVar2783·MATRIX·FP64°changesByOp=_mVar2783·MATRIX·FP64°trainX°trainy°testX°testy°Tr°hpForPruning°changesByOp°changesByPip
    at org.apache.sysds.runtime.controlprogram.Program.execute(Program.java:164)
    at org.apache.sysds.api.ScriptExecutorUtils.executeRuntimeProgram(ScriptExecutorUtils.java:89)
    at org.apache.sysds.api.DMLScript.execute(DMLScript.java:461)
    at org.apache.sysds.api.DMLScript.executeScript(DMLScript.java:304)
    at org.apache.sysds.api.DMLScript.main(DMLScript.java:198)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 1 and 34 -- Error evaluating instruction: CP°fcall°.builtinNS°s_executePipeline°true°12°8°pipeline=_fVar2767·FRAME·FP64·false°Xtrain=_mVar2769·MATRIX·FP64°Ytrain=_mVar2770·MATRIX·FP64°Xtest=_mVar2772·MATRIX·FP64°Ytest=_mVar2773·MATRIX·FP64°metaList=_Var2780·LIST·UNKNOWN·false°hyperParameters=_mVar2782·MATRIX·FP64°flagsCount=5·SCALAR·INT64·true°test=true·SCALAR·BOOLEAN·true°verbose=false·SCALAR·BOOLEAN·true°hpForPruning=_mVar2783·MATRIX·FP64°changesByOp=_mVar2783·MATRIX·FP64°trainX°trainy°testX°testy°Tr°hpForPruning°changesByOp°changesByPip
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:291)
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:204)
    at org.apache.sysds.runtime.controlprogram.BasicProgramBlock.execute(BasicProgramBlock.java:125)
    at org.apache.sysds.runtime.controlprogram.Program.execute(Program.java:158)
    ... 16 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: error executing function .builtinNS::s_executePipeline
    at org.apache.sysds.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:208)
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:255)
    ... 19 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in function program block generated from function statement block between lines 52 and 119 -- Error evaluating function program block
    at org.apache.sysds.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:162)
    at org.apache.sysds.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:201)
    ... 20 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in for program block generated from for statement block between lines 0 and 0 -- Error evaluating for program block
    at org.apache.sysds.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:164)
    at org.apache.sysds.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:155)
    ... 21 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in if program block generated from if statement block between lines 79 and 104 -- Error evaluating if statement body 
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:113)
    at org.apache.sysds.runtime.controlprogram.ForProgramBlock.execute(ForProgramBlock.java:143)
    ... 22 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 80 and 80 -- Error evaluating instruction: CP°eval°outlierByIQR·SCALAR·STRING·true°hp·LIST·UNKNOWN·false°_Var3144·LIST·UNKNOWN
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:291)
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:204)
    at org.apache.sysds.runtime.controlprogram.BasicProgramBlock.execute(BasicProgramBlock.java:125)
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:106)
    ... 23 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: error executing function .builtinNS::m_outlierByIQR
    at org.apache.sysds.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:208)
    at org.apache.sysds.runtime.instructions.cp.EvalNaryCPInstruction.processInstruction(EvalNaryCPInstruction.java:173)
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:255)
    ... 26 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in function program block generated from function statement block between lines 42 and 84 -- Error evaluating function program block
    at org.apache.sysds.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:162)
    at org.apache.sysds.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:201)
    ... 28 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in while program block generated from while statement block between lines 52 and 70 -- Error evaluating while program block
    at org.apache.sysds.runtime.controlprogram.WhileProgramBlock.execute(WhileProgramBlock.java:139)
    at org.apache.sysds.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:155)
    ... 29 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in if program block generated from if statement block between lines 58 and 67 -- Error evaluating if statement body 
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:113)
    at org.apache.sysds.runtime.controlprogram.WhileProgramBlock.execute(WhileProgramBlock.java:117)
    ... 30 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 62 and 62 -- Error evaluating instruction: CP°fcall°.builtinNS°fix_outliers_iqr°false°3°1°X=X·MATRIX·FP64°outlierFilter=outlierFilter·MATRIX·FP64°repairMethod=2·SCALAR·INT64·true°X
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:291)
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:204)
    at org.apache.sysds.runtime.controlprogram.BasicProgramBlock.execute(BasicProgramBlock.java:125)
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:106)
    ... 31 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: error executing function .builtinNS::fix_outliers_iqr
    at org.apache.sysds.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:208)
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:255)
    ... 34 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in function program block generated from function statement block between lines 86 and 106 -- Error evaluating function program block
    at org.apache.sysds.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:162)
    at org.apache.sysds.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:201)
    ... 35 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in if program block generated from if statement block between lines 91 and 103 -- Error evaluating else statement body 
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:126)
    at org.apache.sysds.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:155)
    ... 36 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in if program block generated from if statement block between lines 95 and 103 -- Error evaluating else statement body 
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:126)
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:119)
    ... 37 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in if program block generated from if statement block between lines 97 and 103 -- Error evaluating if statement body 
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:113)
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:119)
    ... 38 more
Caused by: org.apache.sysds.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 99 and 100 -- Error evaluating instruction: CP°replace°pattern=0°replacement=NaN°target=_mVar3267°_mVar3268·MATRIX·FP64
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:291)
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:204)
    at org.apache.sysds.runtime.controlprogram.BasicProgramBlock.execute(BasicProgramBlock.java:125)
    at org.apache.sysds.runtime.controlprogram.IfProgramBlock.execute(IfProgramBlock.java:106)
    ... 39 more
Caused by: java.lang.RuntimeException: Large dense in-memory block (with numblocks=2) allocated but operation access to first block only, which might cause incorrect results.
    at org.apache.sysds.runtime.matrix.data.MatrixBlock.getDenseBlockValues(MatrixBlock.java:565)
    at org.apache.sysds.runtime.matrix.data.MatrixBlock.replaceOperations(MatrixBlock.java:5275)
    at org.apache.sysds.runtime.instructions.cp.ParameterizedBuiltinCPInstruction.processInstruction(ParameterizedBuiltinCPInstruction.java:251)
    at org.apache.sysds.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:255)
    ... 42 more

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

pipeline2.dml
06/Aug/22 14:15
2 kB
Shafaq Siddiqi
sparkDML.sh
06/Aug/22 14:15
0.7 kB
Shafaq Siddiqi
runbug2.sh
06/Aug/22 14:15
0.6 kB
Shafaq Siddiqi

Activity

People

Assignee:: Matthias Boehm

Reporter:: Shafaq Siddiqi

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 06/Aug/22 14:15

Updated:: 15/Mar/23 19:13

Resolved:: 07/Aug/22 13:54