[FLINK-15526] Add RollingPolicy to PartitionWriters

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Table SQL / Ecosystem
• Assignee: Unassigned
• Reporter: Jingsong Lee
• Created: 09/Jan/20 02:40
• Updated: 30/Apr/22 01:25

Description

Now our partition writers will write a whole file in a checkpoint if there is no partition change.

Sometimes this file is too big, it is better to add a RollingPolicy for controlling file size.

We can just add policy controlled by file size and record number.