[YARN-10735] Unmanaged AM is won't populate AMRMToken to ApplicationReport in secure cluster

• Type: Bug
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Assignee: Wang, Xinglong
• Reporter: Wang, Xinglong
• Created: 14/Apr/21 12:17
• Updated: 14/Apr/21 15:34

Description

With kerberos enabled, NPE will be reported when launching UnmanagedAMLauncher.
It is due to there is no AMRMToken is returned in ApplicationReport. After a while investigation, it turns out that RMAppImpl has a bad if condition inside createAndGetApplicationReport

21/04/14 02:46:01 INFO unmanagedamlauncher.UnmanagedAMLauncher: Initializing Client
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Starting Client
21/04/14 02:46:02 INFO client.AHSProxy: Connecting to Application History server at /0.0.0.0:10200
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting up application submission context for ASM
21/04/14 02:46:02 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting unmanaged AM
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Submitting application to ASM
21/04/14 02:46:03 INFO impl.YarnClientImpl: Submitted application application_1618393442264_0002
21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Got application report from ASM for, appId=2, appAttemptId=appattempt_1618393442264_0002_000001, clientToAMToken=Token { kind: YARN_CLIENT_TOKEN, service:  }, appDiagnostics=AM container is launched, waiting for AM container to Register with RM, appMasterHost=N/A, appQueue=abc, appMasterRpcPort=-1, appStartTime=1618393562917, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=abc
21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Launching AM with application attempt id appattempt_1618393442264_0002_000001
21/04/14 02:46:04 FATAL unmanagedamlauncher.UnmanagedAMLauncher: Error running Client
java.lang.NullPointerException
	at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.launchAM(UnmanagedAMLauncher.java:186)
	at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.run(UnmanagedAMLauncher.java:354)
	at org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.main(UnmanagedAMLauncher.java:111)

 

 public ApplicationReport createAndGetApplicationReport(String clientUserName,
      boolean allowAccess) {
        ......
        if (currentAttempt != null && 
            currentAttempt.getAppAttemptState() == RMAppAttemptState.LAUNCHED) {
          if (getApplicationSubmissionContext().getUnmanagedAM() &&
              clientUserName != null && getUser().equals(clientUserName)) {
            Token<AMRMTokenIdentifier> token = currentAttempt.getAMRMToken();
            if (token != null) {
              amrmToken = BuilderUtils.newAMRMToken(token.getIdentifier(),
                  token.getKind().toString(), token.getPassword(),
                  token.getService().toString());
            }
          }
        }

clientUserName is fullName of a kerberos principle like [e-mail] whereas getUser() will return the username recorded in RMAppImpl which is short name.