[MAPREDUCE-5093] Improve RM and HS token acquisition during job submission

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.23.0, 2.0.0-alpha, 3.0.0-alpha1
• Component/s: job submission
• Target Version/s: 2.1.0-beta
• Assignee: Unassigned
• Reporter: Daryn Sharp
• Created: 22/Mar/13 14:50
• Updated: 12/May/16 18:24

Description

Jobs that intend to submit other jobs (ex. oozie, pig) require a RM token. Yarn has added the requirement of a HS token. Currently the submitter is required to explicitly obtain a RM token with the correct renewer and add it to the credentials. To avoid breaking compatibility, the HS token is implicitly acquired if the submitter acquired a RM token via getDelegationToken.

Viewfs exposed the limitations of assuming only one token per filesystem. Similarly, the RM + HS token has the same issue. We should consider changing the api, ex. getDelegationToken(renewer) to addDelegationTokens(renewer, creds) ala the filesystem change.

Further, token acquisition should ideally be considered an internal implementation detail required by security. Submitters, particularly oozie & pig, would benefit greatly from conf setting to indicate jobs are allowed to submit jobs. This conf setting would trigger invoking the proposed addDelegationTokens plus ensure the correct renewer is used, further freeing submitters from knowing internal implementation details of security.

Issue Links

• relates to: MAPREDUCE-5088 MR Client gets an renewer token exception while Oozie is submitting a job