Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.0.1
-
None
-
None
Description
Sometimes we need to identify performance bottlenecks, for example, how long it took to read from data store, how long it took to write into another data store.
It would be great if we can have total duration, read duration, and write duration as task level metrics.
Currently it seems that both `InputMetrics` and `OutputMetrics` do not have duration related metrics.
On the other hand, other metrics such as `ShuffleWriteMetrics` has write time. We might need similar metrics for input/output.