[FLINK-28117] More metrics for FileSource

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.16.0
• Component/s: Connectors / FileSystem, Runtime / Metrics
• Assignee: Unassigned
• Reporter: Jing Ge
• Created: 18/Jun/22 14:39
• Updated: 20/Jun/22 06:39

Description

According to user's requirement[1], following metrics are required:
1. No. of splits SplitAssigner is initialized with, number of splits re-added back to the SplitAssigner
2. Readers created per unit time
3. Time taken to create a reader
4. Time taken for the Reader to produce a single Row
5. Readers closed per unit time
 
Some further nice-to-have metrics:1. Number of rows emitted by the source per unit time
2. Time taken by the enumerator to discover the splits 
3. Total splits discovered
 
Please check FLIP-33 first and extend it if above mentioned metrics were not included in the FLIP.
 
[1] https://lists.apache.org/thread/t70hhss6d9s65y1vygyytbm6sgl05yrl

Issue Links

• is related to: FLINK-28021 Add FLIP-33 metrics to FileSystem connector