Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2928 YARN Timeline Service v.2: alpha 1
  3. YARN-4712

CPU Usage Metric is not captured properly in YARN-2928

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      There are 2 issues with CPU usage collection

      • I was able to observe that that many times CPU usage got from pTree.getCpuUsagePercent() is ResourceCalculatorProcessTree.UNAVAILABLE(i.e. -1) but ContainersMonitor do the calculation i.e. cpuUsageTotalCoresPercentage = cpuUsagePercentPerCore /resourceCalculatorPlugin.getNumProcessors() because of which UNAVAILABLE check in NMTimelinePublisher.reportContainerResourceUsage is not encountered. so proper checks needs to be handled
      • EntityColumnPrefix.METRIC uses always LongConverter but ContainerMonitor is publishing decimal values for the CPU usage.

      Attachments

        1. YARN-4712-YARN-2928.v1.001.patch
          11 kB
          Naganarasimha G R
        2. YARN-4712-YARN-2928.v1.002.patch
          11 kB
          Naganarasimha G R
        3. YARN-4712-YARN-2928.v1.003.patch
          15 kB
          Naganarasimha G R
        4. YARN-4712-YARN-2928.v1.004.patch
          13 kB
          Naganarasimha G R
        5. YARN-4712-YARN-2928.v1.005.patch
          13 kB
          Naganarasimha G R
        6. YARN-4712-YARN-2928.v1.006.patch
          12 kB
          Naganarasimha G R

        Issue Links

          Activity

            People

              Naganarasimha Naganarasimha G R
              Naganarasimha Naganarasimha G R
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: