[FLINK-20671] Partition doesn't commit until the end of partition

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.11.1, 1.12.0
• Component/s: Connectors / FileSystem, Table SQL / Ecosystem
• Assignee: Unassigned
• Reporter: zhuxiaoshang
• Created: 18/Dec/20 07:25
• Updated: 21/Dec/20 08:42
• Resolved: 21/Dec/20 08:42

Description

According to DOC https://ci.apache.org/projects/flink/flink-docs-master/dev/table/connectors/filesystem.html#partition-commit

the partition can be commited multiple times.
Actually, the partition is commited until the end of partition.For example hourly partition, the 10th partition will be commited at 11 clock, even the `sink.partition-commit.delay` is 0s.

In theory,if delay is 0s the partition should be commited on every checkpoint.

Issue Links

• is related to: FLINK-20213 Partition commit is delayed when records keep coming