Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5355 YARN Timeline Service v.2: alpha 2
  3. YARN-6375

App level aggregation should not consider metric values reported in the previous aggregation cycle

    XMLWordPrintableJSON

    Details

      Description

      Currently app level aggregation is done every 15 seconds.
      And we consider last reported metric value for each entity belonging to an app for aggregation.
      We however merely update the corresponding metric values for the entity on put. We never remove the entries.
      But it is possible that multiple entities finish during lifetime of an application. We however continue to consider them till the end.
      We should however not consider metric values of entities unless reported within the 15 second period.

      Consider containers. For a long running app, several containers would start and end at various times during the lifetime of an app.
      To consider metrics for all the containers throughout the lifetime of app, hence wont be correct.

        Attachments

        1. YARN-6375-YARN-5355.01.patch
          8 kB
          Varun Saxena
        2. YARN-6375-YARN-5355.02.patch
          8 kB
          Varun Saxena

          Activity

            People

            • Assignee:
              varun_saxena Varun Saxena
              Reporter:
              varun_saxena Varun Saxena
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: