Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.14.0, 0.14.1, 0.14.2, 0.14.3, 0.14.4, 0.15.0, 0.15.1, 0.15.2, 0.15.3, 0.16.0
-
None
-
None
Description
The exit status of the external processes spawned by streaming tasks is currently logged, but not used to indicate success or failure of the task. While this is reasonable for some UNIX tools (e.g. grep), many programs will indicate failure by a non-zero exit status. (Also, even for custom programs, intentionally indicating the failure of a streaming task is currently rather tricky.)
This could be supported by adding a new job-configuration setting, 'stream.non.zero.exit.is.failure'. If true, a non-zero exit status of a child process would throw an exception in the PipeMapRed, causing task failure. The current behavior would be preserved by using a default setting of false.
This would allow streaming tasks to easily indicate failure, even if all input has already been consumed.