[FLINK-8794] When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.4.0, 1.4.1
• Component/s: Connectors / FileSystem
• Labels: auto-deprioritized-major auto-deprioritized-minor
• Assignee: Unassigned
• Reporter: yanxiaobin
• Created: 27/Feb/18 10:19
• Updated: 09/Dec/21 10:43

Description

When using BucketingSink, it happens that one of the files is always in the [.in-progress] state. And this state has never changed after that.  The underlying use of S3 as storage.

 

// code placeholder

2018-02-28 11:58:42  147341619 _part-28-0.in-progress

2018-02-28 12:06:27  147315059 part-0-0

2018-02-28 12:06:27  147462359 part-1-0

2018-02-28 12:06:27  147316006 part-10-0

2018-02-28 12:06:28  147349854 part-100-0

2018-02-28 12:06:27  147421625 part-101-0

2018-02-28 12:06:27  147443830 part-102-0

2018-02-28 12:06:27  147372801 part-103-0

2018-02-28 12:06:27  147343670 part-104-0

......