Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3342

Runtime profile TotalCpuTime are often wrong

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: impala 2.3
    • Fix Version/s: Impala 2.8.0
    • Component/s: Backend
    • Labels:
      None

      Description

      When tracking scanner thread CPU time, Impala simply uses the wall clock time from thread start to finish. it doesn't eliminate the waiting time. so the TotalCpuTime is much higher than scanner actually use.
      The plan fragment execution CPU time might have similar issue. although it does eliminate disk IO and network transmit time, it might include waiting time for downstream fragments.

        Activity

        Hide
        dhecht Dan Hecht added a comment -

        There is CLOCK_THREAD_CPUTIME_ID that we could use, but it looks to be much slower (probably no vdso implementation): https://upvoid.com/devblog/2014/05/linux-timers/

        Show
        dhecht Dan Hecht added a comment - There is CLOCK_THREAD_CPUTIME_ID that we could use, but it looks to be much slower (probably no vdso implementation): https://upvoid.com/devblog/2014/05/linux-timers/
        Show
        anujphadke Anuj Phadke added a comment - https://github.com/apache/incubator-impala/commit/bb1c63380b8ec14fb6058f3157cad8746463e054

          People

          • Assignee:
            anujphadke Anuj Phadke
            Reporter:
            jyu@cloudera.com Juan Yu
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development