[HDDS-600] Mapreduce example fails with java.lang.IllegalArgumentException: Bucket or Volume name has an unsupported character

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Not A Problem
• Component/s: documentation
• Labels: app-compat test-badlands
• Target Version/s: 0.4.0
• Sprint: HDDS BadLands
• Assignee: Hanisha Koneru
• Reporter: Namit Maheshwari
• Created: 09/Oct/18 07:25
• Updated: 15/Mar/19 21:58
• Resolved: 15/Mar/19 21:58

Description

Set up a hadoop cluster where ozone is also installed. Ozone can be referenced via o3://xx.xx.xx.xx:9889

[root@ctr-e138-1518143905142-510793-01-000002 ~]# ozone sh bucket list o3://xx.xx.xx.xx:9889/volume1/
2018-10-09 07:21:24,624 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[ {
"volumeName" : "volume1",
"bucketName" : "bucket1",
"createdOn" : "Tue, 09 Oct 2018 06:48:02 GMT",
"acls" : [ {
"type" : "USER",
"name" : "root",
"rights" : "READ_WRITE"
}, {
"type" : "GROUP",
"name" : "root",
"rights" : "READ_WRITE"
} ],
"versioning" : "DISABLED",
"storageType" : "DISK"
} ]
[root@ctr-e138-1518143905142-510793-01-000002 ~]# ozone sh key list o3://xx.xx.xx.xx:9889/volume1/bucket1
2018-10-09 07:21:54,500 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[ {
"version" : 0,
"md5hash" : null,
"createdOn" : "Tue, 09 Oct 2018 06:58:32 GMT",
"modifiedOn" : "Tue, 09 Oct 2018 06:58:32 GMT",
"size" : 0,
"keyName" : "mr_job_dir"
} ]
[root@ctr-e138-1518143905142-510793-01-000002 ~]#

Hdfs is also set fine as below

[root@ctr-e138-1518143905142-510793-01-000002 ~]# hdfs dfs -ls /tmp/mr_jobs/input/
Found 1 items
-rw-r--r-- 3 root hdfs 215755 2018-10-09 06:37 /tmp/mr_jobs/input/wordcount_input_1.txt
[root@ctr-e138-1518143905142-510793-01-000002 ~]#

Now try to run Mapreduce example job against ozone o3:

[root@ctr-e138-1518143905142-510793-01-000002 ~]# /usr/hdp/current/hadoop-client/bin/hadoop jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar wordcount /tmp/mr_jobs/input/ o3://xx.xx.xx.xx:9889/volume1/bucket1/mr_job_dir/output
18/10/09 07:15:38 INFO conf.Configuration: Removed undeclared tags:
java.lang.IllegalArgumentException: Bucket or Volume name has an unsupported character : :
at org.apache.hadoop.hdds.scm.client.HddsClientUtils.verifyResourceName(HddsClientUtils.java:143)
at org.apache.hadoop.ozone.client.rpc.RpcClient.getVolumeDetails(RpcClient.java:231)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.ozone.client.OzoneClientInvocationHandler.invoke(OzoneClientInvocationHandler.java:54)
at com.sun.proxy.$Proxy16.getVolumeDetails(Unknown Source)
at org.apache.hadoop.ozone.client.ObjectStore.getVolume(ObjectStore.java:92)
at org.apache.hadoop.fs.ozone.OzoneFileSystem.initialize(OzoneFileSystem.java:121)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3354)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3403)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3371)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:477)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.setOutputPath(FileOutputFormat.java:178)
at org.apache.hadoop.examples.WordCount.main(WordCount.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
18/10/09 07:15:39 INFO conf.Configuration: Removed undeclared tags:
[root@ctr-e138-1518143905142-510793-01-000002 ~]#