[HADOOP-14469] FTPFileSystem#listStatus get currentPath and parentPath at the same time, causing recursively list action endless

• Type: Bug
• Status: Patch Available
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.6.0, 3.0.0-alpha2
• Component/s: fs, tools/distcp

Environment

ftp build by windows7 + Serv-U_64 12.1.0.8
code runs any os

• Assignee: Hongyuan Li
• Reporter: Hongyuan Li
• Created: 31/May/17 07:08
• Updated: 04/Jul/17 11:57

Description

for some ftpsystems, liststatus method will return new Path(".") and new Path(".."), thus causing list op looping.for example, Serv-U
We can see the logic in code below:

  private FileStatus[] listStatus(FTPClient client, Path file)
      throws IOException {
    ……
    FileStatus[] fileStats = new FileStatus[ftpFiles.length];
    for (int i = 0; i < ftpFiles.length; i++) {
      fileStats[i] = getFileStatus(ftpFiles[i], absolute);
    }
    return fileStats;
  }

public void test() throws Exception{
    FTPFileSystem ftpFileSystem = new FTPFileSystem();
    ftpFileSystem.initialize(new Path("ftp://test:123456@192.168.44.1/").toUri(),
        new Configuration());
    FileStatus[] fileStatus  = ftpFileSystem.listStatus(new Path("/new"));
    for(FileStatus fileStatus1 : fileStatus)
      System.out.println(fileStatus1);
}

using test code below, the test results list below

FileStatus{path=ftp://test:123456@192.168.44.1/new; isDirectory=true; modification_time=1496716980000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
FileStatus{path=ftp://test:123456@192.168.44.1/; isDirectory=true; modification_time=1496716980000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
FileStatus{path=ftp://test:123456@192.168.44.1/new/hadoop; isDirectory=true; modification_time=1496716980000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
FileStatus{path=ftp://test:123456@192.168.44.1/new/HADOOP-14431-002.patch; isDirectory=false; length=2036; replication=1; blocksize=4096; modification_time=1495797780000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
FileStatus{path=ftp://test:123456@192.168.44.1/new/HADOOP-14486-001.patch; isDirectory=false; length=1322; replication=1; blocksize=4096; modification_time=1496716980000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
FileStatus{path=ftp://test:123456@192.168.44.1/new/hadoop-main; isDirectory=true; modification_time=1495797120000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}

In results above, FileStatus{path=ftp://test:123456@192.168.44.1/new; …… is obviously the current Path, and
FileStatus{path=ftp://test:123456@192.168.44.1/;…… is obviously the parent Path.
So, if we want to walk the directory recursively, it will stuck.

Issue Links

• requires: HADOOP-14429 FTPFileSystem#getFsAction always returns FsAction.NONE
• requires: HADOOP-14559 FTPFileSystem instance in TestFTPFileSystem should be created before tests and closed after tests