Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9698 [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler
  3. YARN-10099

FS-CS converter: handle allow-undeclared-pools and user-as-default-queue properly and fix misc issues

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      This ticket is intended to fix two issues:

      1. Based on the latest documentation, there are two important properties that are ignored if we have placement rules:

      Property Explanation
      yarn.scheduler.fair.allow-undeclared-pools If this is true, new queues can be created at application submission time, whether because they are specified as the application’s queue by the submitter or because they are placed there by the user-as-default-queue property. If this is false, any time an app would be placed in a queue that is not specified in the allocations file, it is placed in the “default” queue instead. Defaults to true. If a queue placement policy is given in the allocations file, this property is ignored.
      yarn.scheduler.fair.user-as-default-queue Whether to use the username associated with the allocation as the default queue name, in the event that a queue name is not specified. If this is set to “false” or unset, all jobs have a shared default queue, named “default”. Defaults to true. If a queue placement policy is given in the allocations file, this property is ignored.

      Right now these settings affects the conversion regardless of the placement rules.

      2. A converted configuration throws this error:

      2020-01-27 03:35:35,007 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to standby state
      2020-01-27 03:35:35,008 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
      java.lang.IllegalArgumentException: Illegal queue mapping u:%user:%user;u:%user:root.users.%user;u:%user:root.default
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getQueueMappings(CapacitySchedulerConfiguration.java:1113)
      	at org.apache.hadoop.yarn.server.resourcemanager.placement.UserGroupMappingPlacementRule.initialize(UserGroupMappingPlacementRule.java:244)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.getUserGroupMappingPlacementRule(CapacityScheduler.java:671)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.updatePlacementRules(CapacityScheduler.java:712)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:753)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:361)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:426)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
      

      Mapping rules should be separated by a "," character, not by a semicolon.

      3. When initializing FS for conversion, we add the current yarn-site.xml as resource. This is not necessary. This can cause problems like:

      [...]
      20/01/29 02:45:38 ERROR config.RangerConfiguration: addResourceIfReadable(ranger-yarn-audit.xml): couldn't find resource file location
      20/01/29 02:45:38 ERROR config.RangerConfiguration: addResourceIfReadable(ranger-yarn-security.xml): couldn't find resource file location
      20/01/29 02:45:38 ERROR config.RangerConfiguration: addResourceIfReadable(ranger-yarn-policymgr-ssl.xml): couldn't find resource file location
      20/01/29 02:45:38 ERROR conf.Configuration: error parsing conf file:/etc/hadoop/conf.cloudera.YARN-1/xasecure-audit.xml
      java.io.FileNotFoundException: /etc/hadoop/conf.cloudera.YARN-1/xasecure-audit.xml (No such file or directory)
      	at java.base/java.io.FileInputStream.open0(Native Method)
      	at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
      	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
      	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112)
      	at java.base/sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:86)
      	at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:184)
      	at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2966)
      	at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3057)
      	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3018)
      	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
      	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871)
      	at org.apache.hadoop.conf.Configuration.get(Configuration.java:1223)
      	at org.apache.ranger.authorization.hadoop.config.RangerPluginConfig.addSslConfigResource(RangerPluginConfig.java:234)
      	at org.apache.ranger.authorization.hadoop.config.RangerPluginConfig.addResourcesForServiceType(RangerPluginConfig.java:161)
      	at org.apache.ranger.authorization.hadoop.config.RangerPluginConfig.<init>(RangerPluginConfig.java:51)
      	at org.apache.ranger.plugin.service.RangerBasePlugin.<init>(RangerBasePlugin.java:75)
      	at org.apache.ranger.authorization.yarn.authorizer.RangerYarnPlugin.<init>(RangerYarnAuthorizer.java:246)
      	at org.apache.ranger.authorization.yarn.authorizer.RangerYarnAuthorizer.init(RangerYarnAuthorizer.java:81)
      	at org.apache.ranger.authorization.yarn.authorizer.RangerYarnAuthorizer.init(RangerYarnAuthorizer.java:94)
      	at org.apache.hadoop.yarn.security.YarnAuthorizationProvider.getInstance(YarnAuthorizationProvider.java:57)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1375)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1484)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:209)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:102)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)
      20/01/29 02:45:38 WARN config.RangerPluginConfig:  Unable to find SSL Configs
      20/01/29 02:45:38 INFO config.RangerPluginConfig: PolicyEngineOptions: { evaluatorType: auto, evaluateDelegateAdminOnly: false, disableContextEnrichers: false, disableCustomConditions: false, disableTagPolicyEvaluation: false, enableTagEnricherWithLocalRefresher: false, disableTrieLookupPrefilter: false, optimizeTrieForRetrieval: false, cacheAuditResult: false }
      20/01/29 02:45:38 INFO provider.AuditProviderFactory: AuditProviderFactory: creating..
      20/01/29 02:45:38 INFO provider.AuditProviderFactory: AuditProviderFactory: initializing..
      20/01/29 02:45:38 INFO provider.AuditProviderFactory: AuditProviderFactory: Audit not enabled..
      20/01/29 02:45:38 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler failed in state INITED
      java.lang.IllegalArgumentException: bound must be positive
      	at java.base/java.util.Random.nextInt(Random.java:388)
      	at org.apache.ranger.plugin.util.RangerRESTClient.<init>(RangerRESTClient.java:124)
      	at org.apache.ranger.admin.client.RangerAdminRESTClient.init(RangerAdminRESTClient.java:765)
      	at org.apache.ranger.admin.client.RangerAdminRESTClient.init(RangerAdminRESTClient.java:110)
      	at org.apache.ranger.plugin.service.RangerBasePlugin.createAdminClient(RangerBasePlugin.java:584)
      	at org.apache.ranger.plugin.util.PolicyRefresher.<init>(PolicyRefresher.java:95)
      	at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:176)
      	at org.apache.ranger.authorization.yarn.authorizer.RangerYarnPlugin.init(RangerYarnAuthorizer.java:251)
      	at org.apache.ranger.authorization.yarn.authorizer.RangerYarnAuthorizer.init(RangerYarnAuthorizer.java:82)
      	at org.apache.ranger.authorization.yarn.authorizer.RangerYarnAuthorizer.init(RangerYarnAuthorizer.java:94)
      	at org.apache.hadoop.yarn.security.YarnAuthorizationProvider.getInstance(YarnAuthorizationProvider.java:57)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1375)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1484)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:209)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:102)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)
      [...]
      

        Attachments

        1. YARN-10099-010.patch
          28 kB
          Peter Bacsko
        2. YARN-10099-008.patch
          28 kB
          Peter Bacsko
        3. YARN-10099-007.patch
          27 kB
          Peter Bacsko
        4. YARN-10099-006.patch
          25 kB
          Peter Bacsko
        5. YARN-10099-005.patch
          24 kB
          Peter Bacsko
        6. YARN-10099-004.patch
          25 kB
          Peter Bacsko
        7. YARN-10099-003.patch
          25 kB
          Peter Bacsko
        8. YARN-10099-002.patch
          20 kB
          Peter Bacsko
        9. YARN-10099-001.patch
          21 kB
          Peter Bacsko

          Issue Links

            Activity

              People

              • Assignee:
                pbacsko Peter Bacsko
                Reporter:
                pbacsko Peter Bacsko
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: