[YARN-9920] YarnAuthorizationProvider AccessRequest gets Null RemoteAddress from FairScheduler

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.0
• Component/s: fairscheduler, security
• Assignee: Prabhu Joseph
• Reporter: Prabhu Joseph
• Created: 18/Oct/19 18:28
• Updated: 26/Feb/20 05:29

Description

YarnAuthorizationProvider AccessRequest has null RemoteAddress in case of FairScheduler. FSQueue#hasAccess uses Server.getRemoteAddress() which will be null when the call is from RMWebServices and EventDispatcher. It works fine when called by IPC Server Handler.

FSQueue#hasAccess is called at three places where (2) and (3) returns null.

1. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> FSQueue#hasAccess -> Server.getRemoteAddress returns correct Remote IP.

 

2. IPC Server -> RMAppManager#createAndPopulateNewRMApp -> AppAddedSchedulerEvent

    EventDispatcher -> FairScheduler#addApplication -> FSQueue.hasAccess -> Server.getRemoteAddress returns null
 

org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:509)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1268)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:133)
        at org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)

 

3. RMWebServices -> QueueACLsManager#checkAccess -> FSQueue.hasAccess -> Server.getRemoteAddress returns null.

org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer.checkPermission(ConfiguredYarnAuthorizer.java:101)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.hasAccess(FSQueue.java:316)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.checkAccess(FairScheduler.java:1610)
        at org.apache.hadoop.yarn.server.resourcemanager.security.QueueACLsManager.checkAccess(QueueACLsManager.java:84)
        at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.hasAccess(RMWebServices.java:270)
        at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getApps(RMWebServices.java:553)

 

Have verified with CapacityScheduler and it works fine.