[PIG-5180] MergeSparseJoin fails with Spark exec type

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: spark-branch
• Component/s: spark
• Assignee: Ádám Szita
• Reporter: Ádám Szita
• Created: 10/Mar/17 21:08
• Updated: 21/Jun/17 09:18
• Resolved: 13/Mar/17 08:59

Description

MergeSparseJoin 1 to 6 all fail due to following exception being thrown on the frontend side:

Caused by: java.lang.IllegalArgumentException: Can not create a Path from a null string
	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:122)
	at org.apache.hadoop.fs.Path.<init>(Path.java:134)
	at org.apache.pig.backend.hadoop.executionengine.spark.JobGraphBuilder.setReplicationForMergeJoin(JobGraphBuilder.java:126)
	at org.apache.pig.backend.hadoop.executionengine.spark.JobGraphBuilder.visitSparkOp(JobGraphBuilder.java:105)
	at org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkOperator.visit(SparkOperator.java:140)
	at org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkOperator.visit(SparkOperator.java:37)
	at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:87)
	at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:46)
	at org.apache.pig.backend.hadoop.executionengine.spark.SparkLauncher.launchPig(SparkLauncher.java:224)
	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:290)
	... 33 more