Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7069

[Umbrella] YARN Resource Profiles phase#2 enhancements

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      This ticket to is to track further enhancements to resource profile work happening in YARN-3926

        Attachments

          Issue Links

          1.
          Add support for increase and decrease of container resources with resource profiles Sub-task Patch Available Manikandan R
          2.
          Update CapacitySchedulerConfiguration minimum and maximum calculations to consider all resource types Sub-task Patch Available Unassigned
          3.
          Update web UIs to reflect multiple resource types Sub-task Open Varun Vasudev
          4.
          Add support for dynamic resource updates with multiple resource types Sub-task Open Manikandan R
          5.
          Add support for resource profiles in MapReduce Sub-task Patch Available Varun Vasudev
          6.
          Update fair scheduler policies to be aware of resource types Sub-task Resolved Daniel Templeton
          7.
          ResourceTypes should be renamed Sub-task Open Unassigned
          8.
          Node information page in the old web UI should report resource types Sub-task Resolved Gergely Novák
          9.
          Use LightWeightedResource when number of resource types more than two Sub-task Resolved Sunil Govindan
          10.
          Cluster Metrics API should report resource types information Sub-task Open Unassigned
          11.
          Add support for individual resource types requests in MapReduce Sub-task Resolved Gergo Repas
          12.
          Cluster Node API and Cluster Nodes API should report resource types information Sub-task Open Unassigned
          13.
          Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() Sub-task Resolved Manikandan R
          14.
          Clean up different ways to create resources Sub-task Open Unassigned
          15.
          DominantResourceCalculator.isAnyMajorResourceZero() should test all resources Sub-task Resolved Sunil Govindan
          16.
          Improve use of ResourceNotFoundException in resource types code Sub-task Patch Available Daniel Templeton
          17.
          Recover all resources when NM restart Sub-task Resolved Yang Wang
          18.
          TestResourceUtils to be moved to hadoop-yarn-api package Sub-task Open Unassigned
          19.
          FileNotFound handling in ResourceUtils is inconsistent Sub-task Resolved Daniel Templeton
          20.
          Normalize unit of resource objects in RM and avoid to do unit conversion in critical path Sub-task Patch Available Manikandan R
          21.
          Container REST endpoints should report resource types Sub-task Resolved Daniel Templeton
          22.
          Remove class ResourceType Sub-task Resolved Sunil Govindan
          23.
          Support multiple resource types in rmadmin updateNodeResource command Sub-task Resolved Manikandan R
          24.
          Log improvements for the ResourceUtils Sub-task Resolved Sunil Govindan
          25.
          IllegalArgumentException in AMRMClient Sub-task Resolved Unassigned
          26.
          Cleanup usages of ResourceProfiles Sub-task Resolved Wangda Tan
          27.
          Support to specify values of different resource types in DistributedShell for easier testing Sub-task Resolved Gergely Novák
          28.
          Retrospect Resource Profile Behavior for overriding capability Sub-task Resolved Wangda Tan
          29.
          convertToProtoFormat(Resource r) is not setting for all resource types Sub-task Resolved lovekesh bansal
          30.
          Allow client/AM update supported resource types via YARN APIs Sub-task Resolved Sunil Govindan
          31.
          [Resource Profiles] getMaximumAllocationPerQueue to account for all resources Sub-task Patch Available lovekesh bansal
          32.
          Cleaning up the usage of Resources and ResourceCalculator Sub-task Patch Available lovekesh bansal
          33.
          ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores Sub-task Resolved lovekesh bansal
          34.
          Improve the resource types docs Sub-task Resolved Daniel Templeton
          35.
          Improve performance of DRF comparisons for resource types in fair scheduler Sub-task Resolved Daniel Templeton
          36.
          Node resource is not parsed correctly for resource names containing dot Sub-task Resolved Gergely Novák
          37.
          [YARN-7069] Limit format of resource type name Sub-task Resolved Wangda Tan
          38.
          Node manager should allow resource units to be lower cased Sub-task Open Unassigned
          39.
          Node updates don't update the maximum cluster capability for resources other than CPU and memory Sub-task Resolved Daniel Templeton
          40.
          Fair scheduler configuration should allow resource types in the minResources and maxResources properties Sub-task Patch Available Daniel Templeton
          41.
          It should be possible to specify resource types in the fair scheduler increment value Sub-task Resolved Gergo Repas
          42.
          Resource configs not parsed correctly in MapReduce for resources with dot Sub-task Patch Available Jonathan Hung
          43.
          RM gives YARNFeatureNotEnabledException even when resource profile feature is not enabled Sub-task Patch Available Manikandan R
          44.
          Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator Sub-task Resolved Zian Chen
          45.
          Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages. Sub-task Resolved Sunil Govindan
          46.
          Fix UT failure TestRMWebServiceAppsNodelabel#testAppsRunning Sub-task Resolved Sunil Govindan
          47.
          Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy Sub-task Patch Available lovekesh bansal

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                sunilg Sunil Govindan
              • Votes:
                0 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated: