HftpFileSystem.getChecksum(..) does not work for the paths with scheme and authority

• Key: HADOOP-5688
• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Priority: Major
• Assignee: Tsz Wo (Nicholas), SZE
• Reporter: Tsz Wo (Nicholas), SZE
• Fix Version/s: 0.20.1

Description

If a path with scheme and authority information (e.g. hftp://hostname:50070/foo.txt) is passed to HftpFileSystem.getChecksum(..), it won't return the correct file checksum.

Comments

[Tsz Wo (Nicholas), SZE]

The problem is that HftpFileSystem does not construct the url correctly if the path contents scheme and authority information. For example

Path p = new Path("hftp://localhost:2750/foo");
FileSystem hftpfs = p.getFileSystem(conf);
FileChecksum cs = hftpfs.getFileChecksum(p); 
/*
The url constructed above will be
  http://localhost:2750/fileChecksumhftp://localhost:2750/filechecksum/foo?ugi=...
However, the correct url is
  http://localhost:2750/fileChecksum/foo?ugi=...
*/

[Tsz Wo (Nicholas), SZE]

5688_20090415.patch: fix the file checksum url.

[Tsz Wo (Nicholas), SZE]

     [exec] +1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
     [exec] 
     [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.

Running unit tests locally.

[Hadoop QA]

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12405604/5688_20090415.patch
against trunk revision 766670.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 Eclipse classpath. The patch retains Eclipse classpath integrity.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/214/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/214/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/214/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/214/console

This message is automatically generated.

[Chris Douglas]

+1

[Chris Douglas]

I committed this. Thanks, Nicholas

[Hudson]

Integrated in Hadoop-trunk #815 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/815/)
. Fix HftpFileSystem checksum path construction. Contributed by Tsz Wo (Nicholas) Sze