[FLINK-8970] Add more automated end-to-end tests

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Tests
• Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
• Assignee: Unassigned
• Reporter: Till Rohrmann
• Created: 16/Mar/18 10:16
• Updated: 18/Feb/22 10:38

Description

In order to improve Flink's test coverage and make releasing easier, we should add more automated end-to-end tests which test Flink more like a user would interact with the system. Additionally, these end-to-end tests should test the integration of various other systems with Flink.

With FLINK-6539, we added a new module flink-end-to-end-tests which contains the set of currently available end-to-end tests.

With FLINK-8911, a script was added to trigger these tests.

 

This issue is an umbrella issue collecting all different end-to-end tests which we want to add to the Flink repository.

 

Issue Links

• is related to: FLINK-8971 Create general purpose testing job
• is related to: FLINK-8972 Create general purpose DataSet testing job
• links to: GitHub Pull Request #5750

Sub-Tasks

• Introduce automated end-to-end test for local recovery (including sticky scheduling): Closed
• End-to-end test: Run general purpose job with failures in standalone mode: Resolved
• End-to-end test: Run general purpose DataSet job with failures in standalone mode: Closed
• End-to-end test: Resume from savepoint: Closed
• End-to-end test: Resume with different parallelism: Closed
• End-to-end test: Quickstarts: Closed
• End-to-end test: ElasticSearch connector: Closed
• End-to-end test: Manually resume job after terminal failure: Closed
• End-to-end test: Dynamic Kafka partition discovery: Closed
• Extend Kafka end-to-end tests to run with different versions: Closed
• End-to-end test: Stream SQL query with failure and retry: Resolved
• End-to-end test: Job upgrade: Closed
• End-to-end test: BucketingSink: Resolved
• Add end-to-end test for running on YARN with Kerberos: Closed
• End-to-end test: Queryable state: Closed
• End-to-end test: Confluent schema registry: Closed
• End-to-end test: CLI: Resolved
• End-to-end test: REST: In Progress
• End-to-end test: Avro state evolution: Closed
• End-to-end test: Cassandra connector: Closed
• End-to-end test: Ganglia metrics reporter: Open
• Resume from savepoint end-to-end tests should be extended for different state backends: Closed
• End-to-end test: Resume from retained checkpoints: Closed
• End-to-end test: Add test that verifies that a specific classloading issue with avro is fixed: In Progress
• Update `test-ha.sh` end-to-end test to use general purpose DataStream job: Closed
• End-to-end test: Kubernetes integration: Resolved
• End-to-end test: Python API: Closed
• Let externalized checkpoint resume e2e also test rescaling: Resolved
• End-to-end test: Scala Quickstarts: Closed
• End-to-end test: Flink-Example module [flink-examples-streaming]: Open
• End-to-end test: SQL Client with unified source/sink/format: Resolved
• End-to-end test: Streaming job with SSL: Closed
• End-to-end test: Elasticsearch 6.x connector: Resolved
• Add end-to-end test for REST communication encryption: Closed
• Run general purpose test job with failures in per-job mode: Closed
• End-to-end test: Prometheus reporter: Closed
• End-to-end test: Metrics accessible via REST API: Closed
• End-to-end test: Take & resume from savepoint when using per-job mode: Closed
• Add stress deployment end-to-end test: Closed
• Add S3 entropy injection end-to-end/IT test: Closed
• Modify one of the s3 tests to use the Presto S3 s3p schema: Closed
• End-to-end test: Kinesis connector: Resolved
• Add end2end stream sql test for unbounded operators: Open