[MAPREDUCE-6565] Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha2
• Hadoop Flags: Reviewed
• Assignee: Li Lu
• Reporter: Chris Nauroth
• Created: 05/Dec/15 01:03
• Updated: 25/Oct/19 20:25
• Resolved: 30/Nov/16 00:55

Description

By default, the service field of a delegation token is populated based on server IP address. Setting hadoop.security.token.service.use_ip to false changes this behavior to use host name instead of IP address. However, this configuration property is not read from job.xml. Instead, it's read from a separate Configuration instance created during static initialization of SecurityUtil. This does not work correctly with MapReduce jobs if the framework is distributed by setting mapreduce.application.framework.path and the mapreduce.application.classpath is isolated to avoid reading core-site.xml from the cluster nodes. MapReduce tasks will fail to authenticate to HDFS, because they'll try to find a delegation token based on the NameNode IP address, even though at job submission time the tokens were generated using the host name.

Issue Links

• is duplicated by: HADOOP-13173 SecurityUtil.buildtokenService broke token authentication in multi-home network env
• is related to: HADOOP-7733 Mapreduce jobs are failing when JT has hadoop.security.token.service.use_ip=false and client has hadoop.security.token.service.use_ip=true
• relates to: HADOOP-12954 Add a way to change hadoop.security.token.service.use_ip
• relates to: HADOOP-13206 Delegation token cannot be fetched and used by different versions of client