[FLINK-19356] Introduce FileLifeCycleListener to Buckets

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.12.0
• Component/s: Runtime / Task
• Assignee: Jingsong Lee
• Reporter: Jingsong Lee
• Created: 22/Sep/20 10:27
• Updated: 30/Oct/20 02:05
• Resolved: 30/Oct/20 02:05

Description

The interface:

/**
 * Listener about the status of file.
 */
@Internal
public interface FileLifeCycleListener<IN, BucketID> {

/**
 * Notifies a new file has been opened.
 *
 * <p>Note that this does not mean that the file has been created in the file system. It is
 * only created logically and the actual file will be generated after it is committed.
 *
 * @param bucket The bucket of newly opened file.
 * @param newPath The path of newly opened file.
 */
void openPartFile(Bucket<IN, BucketID> bucket, Path newPath);
}

Currently, only supports fileCreated. It can be set to Buckets like BucketLifeCycleListener

FLINK-19345 depends on this to compact files in downstream operator

Issue Links

• blocks: FLINK-19345 In Table File Sink, introduce streaming sink compaction
• links to: GitHub Pull Request #13697