Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9920

YarnAuthorizationProvider AccessRequest gets Null RemoteAddress from FairScheduler

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.3.0
    • Fix Version/s: None
    • Component/s: fairscheduler, security
    • Labels:
      None

      Description

      YarnAuthorizationProvider AccessRequest has null RemoteAddress in case of FairScheduler. FSQueue#hasAccess uses Server.getRemoteAddress() which will be null when the call is from RMWebServices and EventDispatcher. It works fine when called by IPC Server Handler.

      FSQueue#hasAccess is called at three places where (2) and (3) returns null.

      1. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> FSQueue#hasAccess -> Server.getRemoteAddress returns correct Remote IP.

       

      2. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> AppAddedSchedulerEvent

          EventDispatcher -> FairScheduler#addApplication -> FSQueue.hasAccess -> Server.getRemoteAddress returns null
       

      org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:509)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1268)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:133)
              at org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
      

       

      3. RMWebServices -> QueueACLsManager#checkAccess -> FSQueue.hasAccess -> Server.getRemoteAddress returns null.

      org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.checkAccess(FairScheduler.java:1610)
              at org.apache.hadoop.yarn.server.resourcemanager.security.QueueACLsManager.checkAccess(QueueACLsManager.java:84)
              at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.hasAccess(RMWebServices.java:270)
              at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getApps(RMWebServices.java:553)
      

       

      Have verified with CapacityScheduler and it works fine.

        Attachments

        1. AccessAudist_yarn_clientIPempty.png
          159 kB
          Prabhu Joseph
        2. YARN-9920-001.patch
          47 kB
          Prabhu Joseph
        3. YARN-9920-002.patch
          47 kB
          Prabhu Joseph
        4. YARN-9920-003.patch
          48 kB
          Prabhu Joseph
        5. YARN-9920-004.patch
          67 kB
          Prabhu Joseph
        6. YARN-9920-005.patch
          74 kB
          Prabhu Joseph
        7. YARN-9920-006.patch
          74 kB
          Prabhu Joseph

          Activity

            People

            • Assignee:
              prabhujoseph Prabhu Joseph
              Reporter:
              prabhujoseph Prabhu Joseph
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: