[HDFS-8807] dfs.datanode.data.dir does not handle spaces between storageType and URI correctly

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: datanode
• Target Version/s: 2.8.0
• Hadoop Flags: Reviewed
• Assignee: Anu Engineer
• Reporter: Anu Engineer
• Created: 22/Jul/15 20:45
• Updated: 30/Aug/16 01:30
• Resolved: 25/Nov/15 00:06

Description

if you add a space between the storage type and file URI then datanodes fail during startup.
Here is an example of "mis-configration" that leads to datanode failure.

<property>
    <name>dfs.datanode.data.dir</name>
    <value>
      [DISK] file://tmp/hadoop-aengineer/disk1/dfs/data
    </value>
  </property>

Here is the "fixed" version. Please note the lack of space between [DISK] and file URI

<property>
    <name>dfs.datanode.data.dir</name>
    <value>
      [DISK]file://tmp/hadoop-aengineer/disk1/dfs/data
    </value>
  </property>

we fail with a parsing error, here is the info from the datanode logs.

2015-07-22 13:34:36,812 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal character in scheme name at index 0:  file://tmp/hadoop-aengineer/disk1/dfs/data
        at org.apache.hadoop.fs.Path.initialize(Path.java:204)
        at org.apache.hadoop.fs.Path.<init>(Path.java:170)
        at org.apache.hadoop.hdfs.server.datanode.StorageLocation.parse(StorageLocation.java:97)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.getStorageLocations(DataNode.java:2314)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2298)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2349)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2529)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2553)
Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0:  file://tmp/hadoop-aengineer/disk1/dfs/data
        at java.net.URI$Parser.fail(URI.java:2829)
        at java.net.URI$Parser.checkChars(URI.java:3002)
        at java.net.URI$Parser.checkChar(URI.java:3012)
        at java.net.URI$Parser.parse(URI.java:3028)
        at java.net.URI.<init>(URI.java:753)
        at org.apache.hadoop.fs.Path.initialize(Path.java:201)
        ... 7 more