Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4218

Metric for resource*time that was preempted

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: resourcemanager
    • Labels:
      None

      Description

      After YARN-415 we have the ability to track the resource*time footprint of a job and preemption metrics shows how many containers were preempted on a job. However we don't have a metric showing the resource*time footprint cost of preemption. In other words, we know how many containers were preempted but we don't have a good measure of how much work was lost as a result of preemption.
      We should add this metric so we can analyze how much work preemption is costing on a grid and better track which jobs were heavily impacted by it. A job that has 100 containers preempted that only lasted a minute each and were very small is going to be less impacted than a job that only lost a single container but that container was huge and had been running for 3 days.

        Attachments

        1. screenshot-1.png
          33 kB
          Chang Li
        2. screenshot-2.png
          23 kB
          Chang Li
        3. screenshot-3.png
          51 kB
          Chang Li
        4. YARN-4218.006.patch
          58 kB
          Chang Li
        5. YARN-4218.2.patch
          53 kB
          Chang Li
        6. YARN-4218.2.patch
          53 kB
          Chang Li
        7. YARN-4218.2.patch
          53 kB
          Chang Li
        8. YARN-4218.2.patch
          53 kB
          Chang Li
        9. YARN-4218.3.patch
          53 kB
          Chang Li
        10. YARN-4218.4.patch
          53 kB
          Chang Li
        11. YARN-4218.5.patch
          54 kB
          Chang Li
        12. YARN-4218.branch-2.2.patch
          54 kB
          Chang Li
        13. YARN-4218.branch-2.patch
          53 kB
          Chang Li
        14. YARN-4218.patch
          52 kB
          Chang Li
        15. YARN-4218.trunk.2.patch
          51 kB
          Chang Li
        16. YARN-4218.trunk.3.patch
          58 kB
          Chang Li
        17. YARN-4218.trunk.patch
          52 kB
          Chang Li
        18. YARN-4218.wip.patch
          49 kB
          Chang Li
        19. YARN-4218-branch-2.003.patch
          54 kB
          Chang Li

          Issue Links

            Activity

              People

              • Assignee:
                lichangleo Chang Li
                Reporter:
                lichangleo Chang Li
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: