[SYSTEMDS-2353] Parfor optimizer fails in spark exec mode on block partitioning rewrite

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemML 1.2
• Assignee: Matthias Boehm
• Reporter: Matthias Boehm
• Created: 01/Jun/18 03:31
• Updated: 19/Jun/18 01:56
• Resolved: 19/Jun/18 01:56

Description

P = 2
s = 25
Y = rand(rows=100, cols=1, min=1, max=1)
for(i in seq(1,1,1)) {
  n1 = nrow(Y)+0.0
  Y = Y[2:n1,] - Y[1:n1-1,]
}
n = nrow(Y)
Z = rand(rows=n, cols=P, min=0, max=0)
parfor(i2 in seq(1, P, 1)){
  Z[s*i2+1:n,i2] = Y[1:n-s*i2,]
}

Caused by: java.lang.RuntimeException: java.lang.NullPointerException
	at org.apache.sysml.parser.ParForStatementBlock.determineAccessPattern(ParForStatementBlock.java:583)
	at org.apache.sysml.parser.ParForStatementBlock.rDeterminePartitioningCandidates(ParForStatementBlock.java:534)
	at org.apache.sysml.parser.ParForStatementBlock.rDeterminePartitioningCandidates(ParForStatementBlock.java:521)
	at org.apache.sysml.parser.ParForStatementBlock.determineDataPartitionFormat(ParForStatementBlock.java:397)
	at org.apache.sysml.runtime.controlprogram.parfor.opt.OptimizerRuleBased.rewriteSetDataPartitioner(OptimizerRuleBased.java:416)
	at org.apache.sysml.runtime.controlprogram.parfor.opt.OptimizerRuleBased.optimize(OptimizerRuleBased.java:230)
	at org.apache.sysml.runtime.controlprogram.parfor.opt.OptimizationWrapper.optimize(OptimizationWrapper.java:243)
	at org.apache.sysml.runtime.controlprogram.parfor.opt.OptimizationWrapper.optimize(OptimizationWrapper.java:116)
	at org.apache.sysml.runtime.controlprogram.ParForProgramBlock.execute(ParForProgramBlock.java:599)
	at org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:116)