Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3645

ResourceManager can't start success if attribute value of "aclSubmitApps" is null in fair-scheduler.xml

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 3.0.0-alpha2
    • None
    • fairscheduler
    • Add helper method to read configuration field text

    Description

      The "aclSubmitApps" is configured in fair-scheduler.xml like below:
      <queue name="mr">
      <aclSubmitApps></aclSubmitApps>
      </queue>

      The resourcemanager log:

      2015-05-14 12:59:48,623 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager failed in state INITED; cause: org.apache.hadoop.service.ServiceStateException: java.io.IOException: Failed to initialize FairScheduler
      org.apache.hadoop.service.ServiceStateException: java.io.IOException: Failed to initialize FairScheduler
      	at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
      	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:493)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:920)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:240)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1159)
      Caused by: java.io.IOException: Failed to initialize FairScheduler
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1301)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1318)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	... 7 more
      Caused by: java.lang.NullPointerException
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.loadQueue(AllocationFileLoaderService.java:458)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:337)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1299)
      	... 9 more
      2015-05-14 12:59:48,623 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to standby state
      2015-05-14 12:59:48,623 INFO com.zte.zdh.platformplugin.factory.YarnPlatformPluginProxyFactory: plugin transitionToStandbyIn
      2015-05-14 12:59:48,623 WARN org.apache.hadoop.service.AbstractService: When stopping the service ResourceManager : java.lang.NullPointerException
      java.lang.NullPointerException
      	at com.zte.zdh.platformplugin.factory.YarnPlatformPluginProxyFactory.transitionToStandbyIn(YarnPlatformPluginProxyFactory.java:71)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToStandby(ResourceManager.java:997)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStop(ResourceManager.java:1058)
      	at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
      	at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
      	at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:171)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1159)
      2015-05-14 12:59:48,623 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
      org.apache.hadoop.service.ServiceStateException: java.io.IOException: Failed to initialize FairScheduler
      	at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
      	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:493)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:920)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:240)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1159)
      Caused by: java.io.IOException: Failed to initialize FairScheduler
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1301)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1318)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
      	... 7 more
      Caused by: java.lang.NullPointerException
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.loadQueue(AllocationFileLoaderService.java:458)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:337)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1299)
      	... 9 more
      2015-05-14 12:59:48,624 INFO com.zte.zdh.platformplugin.factory.YarnPlatformPluginProxyFactory: plugin shutdown
      2015-05-14 12:59:48,624 ERROR org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread Thread[main,5,main] threw an Exception.
      java.lang.NullPointerException
      	at com.zte.zdh.platformplugin.factory.YarnPlatformPluginProxyFactory.shutdown(YarnPlatformPluginProxyFactory.java:63)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1167)
      2015-05-14 12:59:48,627 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG: 
      

      the reason:
      The method loadQueue in AllocationFileLoaderService.java :

      else if ("aclSubmitApps".equals(field.getTagName())) {
              String text = ((Text)field.getFirstChild()).getData();
              acls.put(QueueACL.SUBMIT_APPLICATIONS, new  AccessControlList(text));
            }
      

      We should check whether getFirstChild() is null.

      Attachments

        1. YARN-3645.1.patch
          14 kB
          Gabor Liptak
        2. YARN-3645.2.patch
          14 kB
          Gabor Liptak
        3. YARN-3645.3.patch
          14 kB
          Gabor Liptak
        4. YARN-3645.4.patch
          14 kB
          Gabor Liptak
        5. YARN-3645.5.patch
          14 kB
          Gabor Liptak
        6. YARN-3645.patch
          6 kB
          Gabor Liptak

        Activity

          People

            gliptak Gabor Liptak
            lyn zhoulinlin
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated: