Details
-
New Feature
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
The goal is to allow users to create side-effect output files on the local disk, and then for successful task attempts, promote the files into an HDFS directory specified by the user. This allows for logging and writing files in formats not easily produced from TRANSFORMs. It might also be nice to have the ability to promote these files regardless of attempt success, which would allow easier debugging of why something failed.