[YARN-8290] SystemMetricsPublisher.appACLsUpdated should be invoked after application information is published to ATS to avoid "User is not set in the application report" Exception

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.1.1
• Fix Version/s: 3.2.0, 3.1.1
• Hadoop Flags: Reviewed
• Assignee: Eric Yang
• Reporter: Yesha Vora
• Created: 15/May/18 00:06
• Updated: 22/May/18 21:13
• Resolved: 22/May/18 20:40

Description

Scenario:

1) Start 5 streaming application in background

2) Kill Active RM and cause RM failover

After RM failover, The application failed with below error.

18/02/01 21:24:29 WARN client.RequestHedgingRMFailoverProxyProvider: Invocation returned exception on [rm2] : org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException: Application with id 'application_1517520038847_0003' doesn't exist in RM. Please check that the job submission was successful.
	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplicationReport(ClientRMService.java:338)
	at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplicationReport(ApplicationClientProtocolPBServiceImpl.java:175)
	at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:417)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347)
, so propagating back to caller.
18/02/01 21:24:29 INFO impl.YarnClientImpl: Submitted application application_1517520038847_0003
18/02/01 21:24:30 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/hrt_qa/.staging/job_1517520038847_0003
18/02/01 21:24:30 ERROR streaming.StreamJob: Error Launching job : User is not set in the application report
Streaming Command Failed!