[HDFS-6667] In HDFS HA mode, Distcp/SLive with webhdfs on secure cluster fails with Client cannot authenticate via:[TOKEN, KERBEROS] error

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.6.0
• Component/s: security
• Hadoop Flags: Reviewed
• Assignee: Jing Zhao
• Reporter: Jian He
• Created: 18/Apr/14 01:14
• Updated: 01/Dec/14 03:08
• Resolved: 17/Jul/14 23:16

Description

Opening on arpitgupta's behalf.

We observed that, in HDFS HA mode, running Distcp/SLive with webhdfs will fail on YARN. In non-HA mode, it'll pass.

The reason is in HA mode, only webhdfs delegation token is generated for the job, but YARN also requires the regular hdfs token to do localization, log-aggregation etc.
In non-HA mode, both tokens are generated for the job.