Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10370

[Umbrella] Reduce the feature gap between FS Placement Rules and CS Queue Mapping rules

    XMLWordPrintableJSON

Details

    Description

      To continue closing the feature gaps between Fair Scheduler and Capacity Scheduler to help users migrate between the scheduler more easy, we need to add some of the Fair Scheduler placement rules to the capacity scheduler's queue mapping functionality.

      With snemeth and pbacsko we've created the following design docs about the proposed changes.

      Attachments

        1.
        		 Create variable context class for CS queue mapping rules Sub-task Resolved Gergely Pollák
        2.
        		 Create MappingRule class to represent each CS mapping rule Sub-task Resolved Gergely Pollák
        3.
        		 Create Matchers for CS mapping rules Sub-task Resolved Gergely Pollák
        4.
        		 Create Actions for CS mapping rules Sub-task Resolved Gergely Pollák
        5.
        		 CS Mapping rule config parser should return MappingRule objects Sub-task Resolved Gergely Pollák
        6.
        		 Create a class that covers the functionality of UserGroupMappingPlacementRule and AppNameMappingPlacementRule using the new mapping rules Sub-task Resolved Gergely Pollák
        7.
        		 Create new JSON schema for Placement Rules Sub-task Resolved Peter Bacsko
        8.
        		 Implement logic which returns MappingRule objects based on mapping rules Sub-task Resolved Peter Bacsko
        9.
        		 Extract MockQueueHierarchyBuilder to a separate class Sub-task Resolved Gergely Pollák
        10.
        		 Improve MockQueueHierarchyBuilder to detect queue ambiguity Sub-task Resolved Gergely Pollák
        11.
        		 Create an allowCreate flag for MappingRuleAction Sub-task Resolved Gergely Pollák
        12.
        		 Move CS placement rule related changes to a separate package Sub-task Resolved Gergely Pollák
        13.
        		 Change fs2cs to generate mapping rules in the new format Sub-task Resolved Peter Bacsko
        14.
        		 Create a group matcher which checks ALL groups of the user Sub-task Resolved Gergely Pollák
        15.
        		 Update CS MappingRule documentation with the new format and features Sub-task Resolved Peter Bacsko
        16.
        		 Adapt existing AppName and UserGroupMapping unittests to ensure backwards compatibility Sub-task Resolved Benjamin Teke
        17.
        		 Replace the legacy placement engine in CS with the new one Sub-task Resolved Gergely Pollák
        18.
        		 Add applicationName policy Sub-task Resolved Peter Bacsko
        19.
        		 Add a configuration switch to change between legacy and JSON placement rule format Sub-task Resolved Gergely Pollák
        20.
        		 Add the capability to fs2cs to write the converted placement rules inside capacity-scheduler.xml Sub-task Resolved Peter Bacsko
        21.
        		 RMAppManager CS Placement ignores parent path Sub-task Resolved Gergely Pollák
        22.
        		 Fix dependency exclusion problem in poms Sub-task Resolved Peter Bacsko
        23.
        		 Create a class which can convert from legacy mapping rule format to the new JSON format Sub-task Resolved Gergely Pollák
        24.
        		 CSMappingPlacementRule should not create new instance of Groups Sub-task Resolved Gergely Pollák
        25.
        		 Fix findbugs issue introduced in YARN-10585 Sub-task Resolved Gergely Pollák
        26.
        		 CS Mapping Rule %specified rule catches default submissions Sub-task Resolved Gergely Pollák
        27.
        		 CSMapping rule can return paths with empty parts Sub-task Resolved Gergely Pollák
        28.
        		 Improve CS MappingRule %secondary_group evaluation Sub-task Resolved Gergely Pollák
        29.
        		 schema generation produces code which doesn't compile on java 11+ Sub-task Resolved Unassigned

        Activity

          People

            shuzirra Gergely Pollák
            shuzirra Gergely Pollák
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: