[FLINK-8973] End-to-end test: Run general purpose job with failures in standalone mode

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.5.0
• Component/s: Tests
• Assignee: Kostas Kloudas
• Reporter: Till Rohrmann
• Created: 16/Mar/18 10:45
• Updated: 03/Apr/18 10:43
• Resolved: 03/Apr/18 10:43

Description

We should set up an end-to-end test which runs the general purpose job (FLINK-8971) in a standalone setting with HA enabled (ZooKeeper). When running the job, the job failures should be activated.

Additionally, we should randomly kill Flink processes (cluster entrypoint and TaskExecutors). When killing them, we should also spawn new processes to make up for the loss.

This end-to-end test case should run with all different state backend settings: RocksDB (full/incremental, async/sync), FsStateBackend (sync/async)

We should then verify that the general purpose job is successfully recovered without data loss or other failures.

Issue Links

• links to: GitHub Pull Request #5750