[FLINK-9322] Add exception throwing map function that simulates failures to the general purpose DataStream job

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.1, 1.6.0
• Component/s: Tests
• Assignee: Tzu-Li (Gordon) Tai
• Reporter: Tzu-Li (Gordon) Tai
• Created: 09/May/18 09:33
• Updated: 22/May/18 08:59
• Resolved: 22/May/18 08:58

Description

The general purpose DataStream job currently does not have any functionality to simulate user job failures.

We can achieve this by:

• Adding a simple-pass map function, that throws exceptions after a certain criteria is met
• To allow for the end-to-end tests that we have in mind, criteria could be to fail after 1) processing X records, and 2) Y completed checkpoints (see FLINK-8977)

• We should also allow specifying how many times to fail. Some chaos monkey tests (see FLINK-8973) would need to continuously fail several times, while FLINK-8977, for example, only needs to fail once.

Issue Links

• links to: GitHub Pull Request #5990