[APEXMALHAR-1160] Create HDFS Output Operator to write with exactly once semantics

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.2
• Assignee: Ashwin Putta
• Reporter: Ashwin Putta
• Created: 02/Jun/14 21:27
• Updated: 03/Jan/16 10:00
• Resolved: 13/Jun/14 20:54

Description

Create a HDFSOutput Operator that writes the data exactly once..
The Operator should create file <window_id>.tmp in the beginwindow and write the tuples to it. It should move the file to <window_id> in the end window. If the operator fails and recover, check if the file <window_id> exists, if it does, then don't process anything during that window, if it doesn't then delete the <window_id>.tmp file if it exists and create new and start writing to it.