This issue suggests error handling as a dispatch layer. For error handling as shims, see T2-1991
When we earlier talked to some users about what options they wanted for error handling, we considered:
a) Removing errors from the list (skipping them)
b) Replacing errors with a replacement input value, e.g. "myStandardGene", empty string, etc.
c) Replacing calls on errors with a replacement output value, e.g. "-- ERROR HERE!!--"
The consideration was not conclusive, but roughly each of these come with issues:
a) Breaks the order of the list - result in position 5 might now come from position 7 in incoming list (5 & 6 were errors) - which makes it hard to use with later dot products
b) Not sure which "fake input" would work well in many cases - and could end up with "fake data" sneaking its way into the results
c) "Error message" values could be passed on to the next service, which don't know it's an error, and falls over with new errors or strange outputs, but these are no longer linked back to original error.
As engineers we thought that we could address c) by giving out the replacement output as "An error with a replacement value". The service which could fail would have an error handling configuration with "[x] Substitute errors with my_value____"
Thus the default for downstream services would be to fail (as of now) - but with a possible error handling configuration to "[x] Allow error replacement values" which would run using the provided replacement value. We were not sure if the outputs of such processors should then be normal values or "errors with replacement values".
We did not proceed with implementing any of these options, as they seemed to require a fair bit of effort and raised more open questions, but issues like T2-1991 shows that users still do need a way to handle errors.
See also wiki: