Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7069

[Umbrella] YARN Resource Profiles phase#2 enhancements

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      This ticket to is to track further enhancements to resource profile work happening in YARN-3926

        Attachments

        Issue Links

        1.
        Add support for increase and decrease of container resources with resource profiles Sub-task Patch Available Manikandan R Actions
        2.
        Update CapacitySchedulerConfiguration minimum and maximum calculations to consider all resource types Sub-task Patch Available Unassigned Actions
        3.
        Update web UIs to reflect multiple resource types Sub-task Open Varun Vasudev Actions
        4.
        Add support for dynamic resource updates with multiple resource types Sub-task Open Manikandan R Actions
        5.
        Add support for resource profiles in MapReduce Sub-task Patch Available Varun Vasudev Actions
        6.
        Update fair scheduler policies to be aware of resource types Sub-task Resolved Daniel Templeton Actions
        7.
        ResourceTypes should be renamed Sub-task Open Zhaohui Xin Actions
        8.
        Node information page in the old web UI should report resource types Sub-task Resolved Gergely Novák Actions
        9.
        Use LightWeightedResource when number of resource types more than two Sub-task Resolved Sunil G Actions
        10.
        Cluster Metrics API should report resource types information Sub-task Open Unassigned Actions
        11.
        Add support for individual resource types requests in MapReduce Sub-task Resolved Gergo Repas Actions
        12.
        Cluster Node API and Cluster Nodes API should report resource types information Sub-task Patch Available Manikandan R Actions
        13.
        Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() Sub-task Resolved Manikandan R Actions
        14.
        Clean up different ways to create resources Sub-task Open Unassigned Actions
        15.
        DominantResourceCalculator.isAnyMajorResourceZero() should test all resources Sub-task Resolved Sunil G Actions
        16.
        Improve use of ResourceNotFoundException in resource types code Sub-task Resolved Szilard Nemeth Actions
        17.
        Recover all resources when NM restart Sub-task Resolved Yang Wang Actions
        18.
        TestResourceUtils to be moved to hadoop-yarn-api package Sub-task Open Zhaohui Xin Actions
        19.
        FileNotFound handling in ResourceUtils is inconsistent Sub-task Resolved Daniel Templeton Actions
        20.
        Normalize unit of resource objects in RM to avoid unit conversion in critical path Sub-task Resolved Manikandan R Actions
        21.
        Container REST endpoints should report resource types Sub-task Resolved Daniel Templeton Actions
        22.
        Remove class ResourceType Sub-task Resolved Sunil G Actions
        23.
        Support multiple resource types in rmadmin updateNodeResource command Sub-task Resolved Manikandan R Actions
        24.
        Log improvements for the ResourceUtils Sub-task Resolved Sunil G Actions
        25.
        IllegalArgumentException in AMRMClient Sub-task Resolved Unassigned Actions
        26.
        Cleanup usages of ResourceProfiles Sub-task Resolved Wangda Tan Actions
        27.
        Support to specify values of different resource types in DistributedShell for easier testing Sub-task Resolved Gergely Novák Actions
        28.
        Retrospect Resource Profile Behavior for overriding capability Sub-task Resolved Wangda Tan Actions
        29.
        convertToProtoFormat(Resource r) is not setting for all resource types Sub-task Resolved lovekesh bansal Actions
        30.
        Allow client/AM update supported resource types via YARN APIs Sub-task Resolved Sunil G Actions
        31.
        [Resource Profiles] getMaximumAllocationPerQueue to account for all resources Sub-task Patch Available lovekesh bansal Actions
        32.
        Cleaning up the usage of Resources and ResourceCalculator Sub-task Patch Available lovekesh bansal Actions
        33.
        ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores Sub-task Resolved lovekesh bansal Actions
        34.
        Improve the resource types docs Sub-task Resolved Daniel Templeton Actions
        35.
        Improve performance of DRF comparisons for resource types in fair scheduler Sub-task Resolved Daniel Templeton Actions
        36.
        Node resource is not parsed correctly for resource names containing dot Sub-task Resolved Gergely Novák Actions
        37.
        [YARN-7069] Limit format of resource type name Sub-task Resolved Wangda Tan Actions
        38.
        Node manager should allow resource units to be lower cased Sub-task Open Zhaohui Xin Actions
        39.
        Node updates don't update the maximum cluster capability for resources other than CPU and memory Sub-task Resolved Daniel Templeton Actions
        40.
        Fair scheduler configuration should allow resource types in the minResources and maxResources properties Sub-task Resolved Szilard Nemeth Actions
        41.
        It should be possible to specify resource types in the fair scheduler increment value Sub-task Resolved Gergo Repas Actions
        42.
        Resource configs not parsed correctly in MapReduce for resources with dot Sub-task Patch Available Jonathan Hung Actions
        43.
        RM gives YARNFeatureNotEnabledException even when resource profile feature is not enabled Sub-task Patch Available Manikandan R Actions
        44.
        Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator Sub-task Resolved Zian Chen Actions
        45.
        Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages. Sub-task Resolved Sunil G Actions
        46.
        Fix UT failure TestRMWebServiceAppsNodelabel#testAppsRunning Sub-task Resolved Sunil G Actions
        47.
        Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy Sub-task Patch Available lovekesh bansal Actions
        48.
        Fair Scheduler stops scheduling when a queue is configured only CPU and memory Sub-task Patch Available Sen Zhao Actions
        49.
        Add resource-types.xml to configuration when refreshing maximum allocation Sub-task Resolved Jonathan Hung Actions

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:

                Issue deployment