[HADOOP-12842] LocalFileSystem checksum file creation fails when source filename contains a colon

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 2.6.4
• Target Version/s: 2.6.5
• Assignee: Plamen Jeliazkov
• Reporter: Plamen Jeliazkov
• Created: 25/Feb/16 19:49
• Updated: 26/Feb/16 08:15
• Resolved: 26/Feb/16 08:15

Description

In most FileSystems you can create a file with a colon character in it, including HDFS. If you try to use the LocalFileSystem implementation (which extends ChecksumFileSystem) to create a file with a colon character in it you get a URISyntaxException during the creation of the checksum file because of the use of

new Path(path, checksumFile)

where checksumFile will be considered as a relative path during URI parsing due to starting with a "." and containing a ":" in the path.

Running the following test inside TestLocalFileSystem causes the failure:

@Test
  public void testColonFilePath() throws Exception {
    FileSystem fs = fileSys;
    Path file = new Path(TEST_ROOT_DIR + Path.SEPARATOR + "fileWith:InIt");
    fs.delete(file, true);
    FSDataOutputStream out = fs.create(file);
    try {
      out.write("text1".getBytes());
    } finally {
      out.close();
    }
}

With the following stack trace:

java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: .fileWith:InIt.crc
	at java.net.URI.checkPath(URI.java:1804)
	at java.net.URI.<init>(URI.java:752)
	at org.apache.hadoop.fs.Path.initialize(Path.java:201)
	at org.apache.hadoop.fs.Path.<init>(Path.java:170)
	at org.apache.hadoop.fs.Path.<init>(Path.java:92)
	at org.apache.hadoop.fs.ChecksumFileSystem.getChecksumFile(ChecksumFileSystem.java:88)
	at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:397)
	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:921)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:902)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:798)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:787)
	at org.apache.hadoop.fs.TestLocalFileSystem.testColonFilePath(TestLocalFileSystem.java:625)

Issue Links

• duplicates: HADOOP-3257 Path should handle all characters