Details
-
Sub-task
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
1.15.0
Description
Documentation of compaction on FileSink: https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/filesystem/#compaction
Possible scenarios might include
- Enable compaction with file-size based compaction strategy.
- Enable compaction with number-checkpoints based compaction strategy.
- Enable compaction, stop-with-savepoint and restarted with compaction disabled.
- Disable compaction, stop-with-savepoint and restarted with compaction enabled.
For each scenario, it might need to verify that
- No repeat and missed records.
- The resulted files' size exceeds the specified condition.