[YARN-10601] The Yarn client should use the UGI who created the Yarn client for obtaining a delegation token for the remote log dir

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Invalid
• Affects Version/s: 3.3.0, 3.4.0
• Component/s: log-aggregation
• Assignee: Unassigned
• Reporter: Daniel Fritsi
• Created: 29/Jan/21 23:29
• Updated: 03/Feb/21 21:47
• Resolved: 03/Feb/21 21:47

Description

It seems there was a bug introduced in YARN-10333 in this section of addLogAggregationDelegationToken:

Path remoteRootLogDir = fileController.getRemoteRootLogDir();
FileSystem fs = remoteRootLogDir.getFileSystem(conf);

final org.apache.hadoop.security.token.Token<?>[] finalTokens =
    fs.addDelegationTokens(masterPrincipal, credentials);

remoteRootLogDir.getFileSystem simply does this:

public FileSystem getFileSystem(Configuration conf) throws IOException {
  return FileSystem.get(this.toUri(), conf);
}

As far as I know it's customary to create a YarnClient instance via YarnClient.createYarnClient() in a UserGroupInformation.doAs block if you would like to use it with a different user then the current one. E.g.:

YarnClient yarnClient = ugi.doAs(new PrivilegedExceptionAction<YarnClient>() {
    @Override
    public YarnClient run() throws Exception {
        YarnClient yarnClient = YarnClient.createYarnClient();
        yarnClient.init(conf);
        yarnClient.start();
        return yarnClient;
    }
});

If this statement is correct then I think YarnClient should save the UserGroupInformation.getCurrentUser() when the YarnClient is being created and the remoteRootLogDir.getFileSystem(conf) call should be made inside an ugi.doAs block with that saved user.

A more concrete example:

public YarnClient createYarnClient(UserGroupInformation ugi, Configuration conf) throws Exception {
    return ugi.doAs((PrivilegedExceptionAction<YarnClient>) () -> {
        // Her I am the submitterUser (see below)
        YarnClient yarnClient = YarnClient.createYarnClient();
        yarnClient.init(conf);
        yarnClient.start();
        return yarnClient;
    });
}

public void run() {
    // Here I am the serviceUser
    // ...
    Configuration conf = ...
    // ...
    UserGroupInformation ugi = getSubmitterUser();
    // ...
    YarnClient yarnClient = createYarnClient(ugi);
    // ...
    ApplicationSubmissionContext context = ...
    // ...
    yarnClient.submitApplication(context);
}

As you can see submitApplication is not invoked inside an ugi.doAs block and submitApplication is the one who will eventually invoke addLogAggregationDelegationToken. That's why we need to save the UGI during the YarnClient creation and create the FileSystem instance inside an ugi.doAs with that saved user. Otherwise Yarn will try to get a delegation token with an incorrect user (serviceUser) instead of the submitterUser.

Issue Links

• is caused by: YARN-10333 YarnClient obtain Delegation Token for Log Aggregation Path