[MRQL-28] Fixed dumping of HDFS sequence file in Spark mode

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Component/s: Run-Time/Spark
• Assignee: Leonidas Fegaras
• Reporter: Leonidas Fegaras
• Created: 02/Feb/14 21:00
• Updated: 04/Feb/14 18:12
• Resolved: 04/Feb/14 18:12

Description

The attached patch fixes the following bug in Spark mode:
When using the MRQL "store file from e" in Spark distributed mode and the file is already in HDFS, it reports an error instead of replacing the old file with the new.
This is because the Spark method saveAsHadoopFile creates a file but cannot replace an existing HDFS file (this may be a Spark bug). The patch simply deletes the file before it calls the saveAsHadoopFile method.