Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
While running the cleaning pipelines in Spark context the script terminated with following exception,
An Error Occured : DMLRuntimeException -- org.apache.sysds.runtime.DMLRuntimeException: PARFOR: Failed to execute loop in parallel. DMLRuntimeException -- PARFOR: Failed to execute loop in parallel. DMLRuntimeException -- Not supported: Instructions of type other than CP instructions org.apache.sysds.runtime.instructions.spark.MultiReturnParameterizedBuiltinSPInstruction SPARK°transformencode°F·FRAME·STRING·false°_Var5410935·SCALAR·STRING·false°X°M org.apache.sysds.runtime.DMLRuntimeException: org.apache.sysds.runtime.DMLRuntimeException: PARFOR: Failed to execute loop in parallel. at org.apache.sysds.runtime.controlprogram.Program.execute(Program.java:155) at org.apache.sysds.api.ScriptExecutorUtils.executeRuntimeProgram(ScriptExecutorUtils.java:88) at org.apache.sysds.api.DMLScript.execute(DMLScript.java:423) at org.apache.sysds.api.DMLScript.executeScript(DMLScript.java:266) at org.apache.sysds.api.DMLScript.main(DMLScript.java:164) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: org.apache.sysds.runtime.DMLRuntimeException: PARFOR: Failed to execute loop in parallel. at org.apache.sysds.runtime.controlprogram.ParForProgramBlock.execute(ParForProgramBlock.java:649) at org.apache.sysds.runtime.controlprogram.Program.execute(Program.java:149) ... 16 more Caused by: org.apache.sysds.runtime.DMLRuntimeException: Not supported: Instructions of type other than CP instructions org.apache.sysds.runtime.instructions.spark.MultiReturnParameterizedBuiltinSPInstruction SPARK°transformencode°F·FRAME·STRING·false°_Var5410935·SCALAR·STRING·false°X°M at org.apache.sysds.runtime.util.ProgramConverter.serializeInstructions(ProgramConverter.java:995) at org.apache.sysds.runtime.util.ProgramConverter.rSerializeProgramBlock(ProgramConverter.java:1229) at org.apache.sysds.runtime.util.ProgramConverter.rSerializeProgramBlocks(ProgramConverter.java:1118) at org.apache.sysds.runtime.util.ProgramConverter.rSerializeProgramBlock(ProgramConverter.java:1222) at org.apache.sysds.runtime.util.ProgramConverter.rSerializeFunctionProgramBlocks(ProgramConverter.java:1104) at org.apache.sysds.runtime.util.ProgramConverter.serializeProgram(ProgramConverter.java:858) at org.apache.sysds.runtime.util.ProgramConverter.serializeParForBody(ProgramConverter.java:820) at org.apache.sysds.runtime.controlprogram.ParForProgramBlock.executeRemoteSparkParFor(ParForProgramBlock.java:853) at org.apache.sysds.runtime.controlprogram.ParForProgramBlock.execute(ParForProgramBlock.java:637) ... 17 more