Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
None
Description
Write HDFS Output adapter which is able to write the incoming tuples to HDFS following the transaction semantics.
All the tuples in a given window should be written to a file identified by user provided file (preferably format which takes into account the application id, operator id, window id, operator name etc).
The file should be flushed/closed during endWindow depending upon whether the name changes from one end window to the next one.
The current HDFSOutputAdapter is practically not very useful. It can be a specific extension of abstract operator and should be implemented so or deleted from Malhar.