Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2928 YARN Timeline Service v.2: alpha 1
  3. YARN-4712

CPU Usage Metric is not captured properly in YARN-2928

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      There are 2 issues with CPU usage collection

      • I was able to observe that that many times CPU usage got from pTree.getCpuUsagePercent() is ResourceCalculatorProcessTree.UNAVAILABLE(i.e. -1) but ContainersMonitor do the calculation i.e. cpuUsageTotalCoresPercentage = cpuUsagePercentPerCore /resourceCalculatorPlugin.getNumProcessors() because of which UNAVAILABLE check in NMTimelinePublisher.reportContainerResourceUsage is not encountered. so proper checks needs to be handled
      • EntityColumnPrefix.METRIC uses always LongConverter but ContainerMonitor is publishing decimal values for the CPU usage.

        Attachments

        1. YARN-4712-YARN-2928.v1.001.patch
          11 kB
          Naganarasimha G R
        2. YARN-4712-YARN-2928.v1.002.patch
          11 kB
          Naganarasimha G R
        3. YARN-4712-YARN-2928.v1.003.patch
          15 kB
          Naganarasimha G R
        4. YARN-4712-YARN-2928.v1.004.patch
          13 kB
          Naganarasimha G R
        5. YARN-4712-YARN-2928.v1.005.patch
          13 kB
          Naganarasimha G R
        6. YARN-4712-YARN-2928.v1.006.patch
          12 kB
          Naganarasimha G R

          Issue Links

            Activity

              People

              • Assignee:
                Naganarasimha Naganarasimha G R
                Reporter:
                Naganarasimha Naganarasimha G R
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: