Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6592 [Umbrella] Rich placement constraints in YARN
  3. YARN-7784

Fix Cluster metrics when placement processor is enabled

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0
    • Component/s: metrics, RM
    • Labels:
      None
    • Target Version/s:

      Description

      Reproducing steps

      1. Setup a cluster and sets yarn.resourcemanager.placement-constraints.enabled to true
      2. Submit a DS job with placement constraint, such as -placement_spec foo=2,NOTIN,NODE,foo
      3. Check cluster metrics from http://<RM_ADDRESS>/cluster/apps

      when job is running, Containers Running, Memory Used and VCore Used were not updated (except AM), metrics from containers allocated by the PlacementProcessor were not accumulated to the cluster metrics; however when job is done, the resource were deducted. Then UI displays like following:

      • Containers Running: -2
      • Memory Used: -400
      • VCores Used: -2

      Looks like AppSchedulingInfo#updateMetricsForAllocatedContainer was not called when allocating a container.

        Attachments

        1. YARN-7784-YARN-6592.001.patch
          9 kB
          Arun Suresh

          Activity

            People

            • Assignee:
              asuresh Arun Suresh
              Reporter:
              cheersyang Weiwei Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: