[FLINK-12521] Use minio as AWS S3 alternative in end-to-end tests

• Type: Improvement
• Status: Reopened
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: FileSystems, Tests
• Assignee: Unassigned
• Reporter: Alex
• Created: 15/May/19 13:22
• Updated: 03/Mar/22 12:56

Description

Some end-to-end tests (located in flink-end-to-end-tests/test-scripts) are using S3 storage for testing.
It may be convenient to use MinIO (or if there are other alternatives) as drop in replacement to run tests locally (or in CI environment).

A test MinIO service lifetime would be bound to a single test execution (in similar way as some tests already spin up/tear down Zookeeper and Kafka clusters locally).

The tests should be runnable against both alternatives by a simple configuration switch.

Issue Links

• is duplicated by: FLINK-26087 S3 filesystem tests could be enabled to run on PR AzureCI builds as well