Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16864

Add idle metrics for Task

    XMLWordPrintableJSON

    Details

    • Release Note:
      Added idleTime metric for Tasks

      Description

      Currently there is no metric for user to measure how busy a task is concretely, which is important for user to decide how to tune a job.

      We would like to propose adding an IdleTime which measure idle time of a task including the time cost for mail processor to wait for new mail and the time cost in record writer to waiting a new buffer.

      With the idle time:
      1. when a job can not catch up with the speed of data generating, the vertex which idle time is near to zero is the bottle neck of the job.
      2. when a job is not busy, idle time can be used to guide user how much he can scale down the job.

      In addition, measuring idle time can have little impaction on the performance of the job, because when a task is busy, we don't touch the code to measure wait-time in mailbox.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                wenlong.lwl Wenlong Lyu
                Reporter:
                wenlong.lwl Wenlong Lyu
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m