Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.4.2
-
None
Description
When looking at the task metrics for running tasks in the task table for the related stage, the duration column is not updated until the task has succeeded. The duration values are reported empty or 0 ms until the task has completed. This is a change in behavior, from earlier versions, when the task duration was continuously updated while the task was running. The missing duration values can be observed for both short and long running tasks and for multiple applications.
To reproduce this, one can run any code from the spark-shell and observe the missing duration values for any running task. Only when the task succeeds is the duration value populated in the UI.
Attachments
Attachments
Issue Links
- links to