Blocker Description
In order to verify the BucketingSink, we should add an end-to-end test which verifies that the BucketingSink does not lose data under failures.
An idea would be to have a CountUp job which simply counts up a counter which is persisted. The emitted values will be written to disk by the BucketingSink. Now we should kill randomly Flink processes (cluster entrypoint and TaskExecutors) to simulate failures. Even after these failures, the written files should contain the correct sequence of numbers.
Attachments
Issue Links
- links to
