Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9322

Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      YARN-8842 implemented storing and exposing of metrics of custom resources.
      FSQueueMetrics should have a similar implementation.
      All metrics stored in this class should have their custom resource counterpart.

      In a consequence of metrics were not stored for custom resource type, FairSchedulerQueueInfo haven't contained those values therefore the UI v1 could not show them, obviously.
      See that gpu is missing from the value of "AM Max Resources" on the attached screenshot.

      Additionally, the callees of the following methods (in class FairSchedulerQueueInfo) should consider to query values for custom resource types too:
      getMaxAMShareMB
      getMaxAMShareVCores
      getAMResourceUsageMB
      getAMResourceUsageVCores

        Attachments

        1. YARN-9322.006.patch
          35 kB
          Szilard Nemeth
        2. YARN-9322.005.patch
          44 kB
          Szilard Nemeth
        3. YARN-9322.004.patch
          29 kB
          Szilard Nemeth
        4. YARN-9322.003.patch
          35 kB
          Szilard Nemeth
        5. YARN-9322.002.patch
          29 kB
          Szilard Nemeth
        6. YARN-9322.001.patch
          30 kB
          Szilard Nemeth
        7. Screen Shot 2019-02-21 at 12.06.46.png
          353 kB
          Szilard Nemeth

          Issue Links

            Activity

              People

              • Assignee:
                snemeth Szilard Nemeth
                Reporter:
                snemeth Szilard Nemeth
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: