[FLINK-33856] Add sub-task spans to TraceReporter for checkpointing

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.18.0
• Component/s: Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Jufang He
• Reporter: Jufang He
• Created: 15/Dec/23 09:34
• Updated: 24/Apr/24 08:09

Description

When Flink makes a checkpoint, the interaction performance with the external file system has a great impact on the overall time-consuming. Therefore, it is easy to observe the bottleneck point by adding performance indicators when the task interacts with the external file storage system. These include: the rate of file write , the latency to write the file, the latency to close the file.

In flink side add the above metrics has the following advantages: convenient statistical different task E2E time-consuming; do not need to distinguish the type of external storage system, can be unified in the FsCheckpointStreamFactory.

Issue Links

• links to: GitHub Pull Request #23989

Sub-Tasks

• Add checkpoint metrics: the rate of file write: Open
• Add checkpoint metrics: the latency to write the file: Open
• Add checkpoint metrics: the latency to close the file: Open