Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12842

LocalFileSystem checksum file creation fails when source filename contains a colon

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 2.6.4
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      In most FileSystems you can create a file with a colon character in it, including HDFS. If you try to use the LocalFileSystem implementation (which extends ChecksumFileSystem) to create a file with a colon character in it you get a URISyntaxException during the creation of the checksum file because of the use of

      new Path(path, checksumFile)

      where checksumFile will be considered as a relative path during URI parsing due to starting with a "." and containing a ":" in the path.

      Running the following test inside TestLocalFileSystem causes the failure:

      @Test
        public void testColonFilePath() throws Exception {
          FileSystem fs = fileSys;
          Path file = new Path(TEST_ROOT_DIR + Path.SEPARATOR + "fileWith:InIt");
          fs.delete(file, true);
          FSDataOutputStream out = fs.create(file);
          try {
            out.write("text1".getBytes());
          } finally {
            out.close();
          }
      }
      

      With the following stack trace:

      java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: .fileWith:InIt.crc
      	at java.net.URI.checkPath(URI.java:1804)
      	at java.net.URI.<init>(URI.java:752)
      	at org.apache.hadoop.fs.Path.initialize(Path.java:201)
      	at org.apache.hadoop.fs.Path.<init>(Path.java:170)
      	at org.apache.hadoop.fs.Path.<init>(Path.java:92)
      	at org.apache.hadoop.fs.ChecksumFileSystem.getChecksumFile(ChecksumFileSystem.java:88)
      	at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:397)
      	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
      	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:921)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:902)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:798)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:787)
      	at org.apache.hadoop.fs.TestLocalFileSystem.testColonFilePath(TestLocalFileSystem.java:625)
      

        Attachments

        1. HADOOP-12842_trunk.patch
          2 kB
          Plamen Jeliazkov

          Issue Links

            Activity

              People

              • Assignee:
                zero45 Plamen Jeliazkov
                Reporter:
                zero45 Plamen Jeliazkov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 24h
                  24h
                  Remaining:
                  Remaining Estimate - 24h
                  24h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified