[HADOOP-17011] Tolerate leading and trailing spaces in fs.defaultFS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: common
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Ctest
• Reporter: Ctest
• Created: 23/Apr/20 21:56
• Updated: 11/Feb/24 12:32
• Resolved: 30/Apr/20 21:27

Description

Problem:

Currently, `getDefaultUri` is using `conf.get` to get the value of `fs.defaultFS`, which means that the trailing whitespace after a valid URI won’t be removed and could stop namenode and datanode from starting up.

 

How to reproduce (Hadoop-2.8.5):

Set the configuration

<property>
     <name>fs.defaultFS</name>
     <value>hdfs://localhost:9000 </value>
</property>

In core-site.xml (there is a whitespace after 9000) and start HDFS.

Namenode and datanode won’t start and the log message is:

2020-04-23 11:09:48,198 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
java.lang.IllegalArgumentException: Illegal character in authority at index 7: hdfs://localhost:9000 
    at java.net.URI.create(URI.java:852)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.setClientNamenodeAddress(NameNode.java:440)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:897)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:885)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694)
Caused by: java.net.URISyntaxException: Illegal character in authority at index 7: hdfs://localhost:9000 
    at java.net.URI$Parser.fail(URI.java:2848)
    at java.net.URI$Parser.parseAuthority(URI.java:3186)
    at java.net.URI$Parser.parseHierarchical(URI.java:3097)
    at java.net.URI$Parser.parse(URI.java:3053)
    at java.net.URI.<init>(URI.java:588)
    at java.net.URI.create(URI.java:850)
    ... 5 more

 

Solution:

Use `getTrimmed` instead of `get` for `fs.defaultFS`:

public static URI getDefaultUri(Configuration conf) {
  URI uri =
    URI.create(fixName(conf.getTrimmed(FS_DEFAULT_NAME_KEY, DEFAULT_FS)));
  if (uri.getScheme() == null) {
    throw new IllegalArgumentException("No scheme in default FS: " + uri);
  }
  return uri;
}

I have submitted a patch for trunk about this.