[FLINK-26322] Test FileSink compaction manually

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.15.0
• Component/s: Connectors / FileSystem
• Labels: release-testing
• Assignee: Alexander Preuss
• Reporter: Yun Gao
• Created: 23/Feb/22 07:55
• Updated: 22/Mar/22 12:35
• Resolved: 14/Mar/22 09:30

Description

Documentation of compaction on FileSink: https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/filesystem/#compaction

Possible scenarios might include

1. Enable compaction with file-size based compaction strategy.
2. Enable compaction with number-checkpoints based compaction strategy.
3. Enable compaction, stop-with-savepoint and restarted with compaction disabled.
4. Disable compaction, stop-with-savepoint and restarted with compaction enabled.

For each scenario, it might need to verify that

1. No repeat and missed records.
2. The resulted files' size exceeds the specified condition.