Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10370

[Umbrella] Reduce the feature gap between FS Placement Rules and CS Queue Mapping rules

    XMLWordPrintableJSON

    Details

      Description

      To continue closing the feature gaps between Fair Scheduler and Capacity Scheduler to help users migrate between the scheduler more easy, we need to add some of the Fair Scheduler placement rules to the capacity scheduler's queue mapping functionality.

      With Szilard Nemeth and Peter Bacsko we've created the following design docs about the proposed changes.

        Attachments

        1.
        		 Create variable context class for CS queue mapping rules Sub-task Resolved Gergely Pollak
        2.
        		 Create MappingRule class to represent each CS mapping rule Sub-task Resolved Gergely Pollak
        3.
        		 Create Matchers for CS mapping rules Sub-task Resolved Gergely Pollak
        4.
        		 Create Actions for CS mapping rules Sub-task Resolved Gergely Pollak
        5.
        		 CS Mapping rule config parser should return MappingRule objects Sub-task Resolved Gergely Pollak
        6.
        		 Create a class that covers the functionality of UserGroupMappingPlacementRule and AppNameMappingPlacementRule using the new mapping rules Sub-task Resolved Gergely Pollak
        7.
        		 Create new JSON schema for Placement Rules Sub-task Resolved Peter Bacsko
        8.
        		 Implement logic which returns MappingRule objects based on mapping rules Sub-task Resolved Peter Bacsko
        9.
        		 Extract MockQueueHierarchyBuilder to a separate class Sub-task Resolved Gergely Pollak
        10.
        		 Improve MockQueueHierarchyBuilder to detect queue ambiguity Sub-task Resolved Gergely Pollak
        11.
        		 Create an allowCreate flag for MappingRuleAction Sub-task Resolved Gergely Pollak
        12.
        		 Move CS placement rule related changes to a separate package Sub-task Resolved Gergely Pollak
        13.
        		 Change fs2cs to generate mapping rules in the new format Sub-task Resolved Peter Bacsko
        14.
        		 Create a group matcher which checks ALL groups of the user Sub-task Resolved Gergely Pollak
        15.
        		 Update CS MappingRule documentation with the new format and features Sub-task Resolved Peter Bacsko
        16.
        		 Adapt existing AppName and UserGroupMapping unittests to ensure backwards compatibility Sub-task Resolved Benjamin Teke
        17.
        		 Replace the legacy placement engine in CS with the new one Sub-task Resolved Gergely Pollak
        18.
        		 Add applicationName policy Sub-task Resolved Peter Bacsko
        19.
        		 Add a configuration switch to change between legacy and JSON placement rule format Sub-task Resolved Gergely Pollak
        20.
        		 Add the capability to fs2cs to write the converted placement rules inside capacity-scheduler.xml Sub-task Resolved Peter Bacsko
        21.
        		 RMAppManager CS Placement ignores parent path Sub-task Resolved Gergely Pollak
        22.
        		 Fix dependency exclusion problem in poms Sub-task Resolved Peter Bacsko
        23.
        		 Create a class which can convert from legacy mapping rule format to the new JSON format Sub-task Resolved Gergely Pollak
        24.
        		 CSMappingPlacementRule should not create new instance of Groups Sub-task Resolved Gergely Pollak
        25.
        		 Fix findbugs issue introduced in YARN-10585 Sub-task Resolved Gergely Pollak
        26.
        		 CS Mapping Rule %specified rule catches default submissions Sub-task Resolved Gergely Pollak
        27.
        		 CSMapping rule can return paths with empty parts Sub-task Resolved Gergely Pollak
        28.
        		 Improve CS MappingRule %secondary_group evaluation Sub-task Resolved Gergely Pollak

          Activity

            People

            • Assignee:
              shuzirra Gergely Pollak
              Reporter:
              shuzirra Gergely Pollak
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: