[PIG-4605] fix a bug when coping Jar to SparkJob working directory

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: spark-branch
• Component/s: spark
• Assignee: Xianda Ke
• Reporter: Xianda Ke
• Created: 16/Jun/15 07:47
• Updated: 21/Jun/17 09:18
• Resolved: 16/Jun/15 13:08

Description

The function SparkLauncher.addJarToSparkJobWorkingDirectory() try to copy the user's Jar files to the working directory.

when SPARK_MATER == LOCAL, and if the src & dest Jar files have the same absolute path, an exception will be thrown.

Reproduce steps:
1. build pig tutorial
2. cd PIG_HOME/tutorial/pigtmp
pig -x spark
REGISTER ./tutorial.jar
raw = LOAD '/user/excite-small.log' USING PigStorage('\t') AS (user, time, query);
clean1 = FILTER raw BY org.apache.pig.tutorial.NonURLDetector(query);
store clean1 into '/user/result1';