[OOZIE-3035] HDFS HA and log aggregation: getting HDFS delegation token from YARN renewer within JavaActionExecutor

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.3.0
• Fix Version/s: 5.0.0b1

Environment

• Hadoop 3 alpha 4
• HDFS HA
• log aggregation turned on

• Assignee: Andras Piros
• Reporter: Andras Piros
• Created: 14/Aug/17 14:23
• Updated: 25/Jan/18 21:00
• Resolved: 31/Aug/17 13:30

Description

In a secure environment, when both HDFS HA and log aggregation are turned on, JavaActionExecutor is not able to call YarnClient#submitApplication since HDFS_DELEGATION_TOKEN is missing.

In those cases we need to get HDFS_DELEGATION_TOKEN from YARN:

• get server principal / YARN renewer via HadoopAccessorService#getServerPrincipal
• get HDFS_DELEGATION_TOKEN via DFSClient#getDelegationToken
• add HDFS_DELEGATION_TOKEN to Credentials

Issue Links

• links to: ReviewBoard link