[HDFS-3062] Fail to submit mapred job on a secured-HA-HDFS: logic URI cannot be picked up by job submission.

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 2.0.0-alpha
• Component/s: ha, security
• Target Version/s: 0.23.3
• Hadoop Flags: Reviewed
• Assignee: Mingjie Lai
• Reporter: Mingjie Lai
• Created: 08/Mar/12 00:58
• Updated: 28/Sep/15 20:58
• Resolved: 16/Mar/12 00:01

Description

When testing the combination of NN HA + security + yarn, I found that the mapred job submission cannot pick up the logic URI of a nameservice.

I have logic URI configured in core-site.xml

<property>
 <name>fs.defaultFS</name>
 <value>hdfs://ns1</value>
</property>

HDFS client can work with the HA deployment/configs:

[root@nn1 hadoop]# hdfs dfs -ls /
Found 6 items
drwxr-xr-x   - hbase  hadoop          0 2012-03-07 20:42 /hbase
drwxrwxrwx   - yarn   hadoop          0 2012-03-07 20:42 /logs
drwxr-xr-x   - mapred hadoop          0 2012-03-07 20:42 /mapred
drwxr-xr-x   - mapred hadoop          0 2012-03-07 20:42 /mr-history
drwxrwxrwt   - hdfs   hadoop          0 2012-03-07 21:57 /tmp
drwxr-xr-x   - hdfs   hadoop          0 2012-03-07 20:42 /user

but cannot submit a mapred job with security turned on

[root@nn1 hadoop]# /usr/lib/hadoop/bin/yarn --config ./conf jar share/hadoop/mapreduce/hadoop-mapreduce-examples-0.24.0-SNAPSHOT.jar randomwriter out
Running 0 maps.
Job started: Wed Mar 07 23:28:23 UTC 2012
java.lang.IllegalArgumentException: java.net.UnknownHostException: ns1
	at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:431)
	at org.apache.hadoop.security.SecurityUtil.buildDTServiceName(SecurityUtil.java:312)
	at org.apache.hadoop.fs.FileSystem.getCanonicalServiceName(FileSystem.java:217)
	at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:119)
	at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:97)
	at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
	at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
	at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:411)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:326)
	at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1221)
	at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
....

0.24

Issue Links

• blocks: HDFS-3083 Cannot run an MR job with HA and security enabled when second-listed NN active
• incorporates: HDFS-5103 TestDirectoryScanner fails on Windows