Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14469

FTPFileSystem#listStatus get currentPath and parentPath at the same time, causing recursively list action endless

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 2.6.0, 3.0.0-alpha2
    • Fix Version/s: None
    • Component/s: fs, tools/distcp
    • Labels:
      None
    • Environment:

      ftp build by windows7 + Serv-U_64 12.1.0.8
      code runs any os

      Description

      for some ftpsystems, liststatus method will return new Path(".") and new Path(".."), thus causing list op looping.for example, Serv-U
      We can see the logic in code below:

        private FileStatus[] listStatus(FTPClient client, Path file)
            throws IOException {
          ……
          FileStatus[] fileStats = new FileStatus[ftpFiles.length];
          for (int i = 0; i < ftpFiles.length; i++) {
            fileStats[i] = getFileStatus(ftpFiles[i], absolute);
          }
          return fileStats;
        }
      
      public void test() throws Exception{
          FTPFileSystem ftpFileSystem = new FTPFileSystem();
          ftpFileSystem.initialize(new Path("ftp://test:123456@192.168.44.1/").toUri(),
              new Configuration());
          FileStatus[] fileStatus  = ftpFileSystem.listStatus(new Path("/new"));
          for(FileStatus fileStatus1 : fileStatus)
            System.out.println(fileStatus1);
      }
      

      using test code below, the test results list below

      FileStatus{path=ftp://test:123456@192.168.44.1/new; isDirectory=true; modification_time=1496716980000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
      FileStatus{path=ftp://test:123456@192.168.44.1/; isDirectory=true; modification_time=1496716980000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
      FileStatus{path=ftp://test:123456@192.168.44.1/new/hadoop; isDirectory=true; modification_time=1496716980000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
      FileStatus{path=ftp://test:123456@192.168.44.1/new/HADOOP-14431-002.patch; isDirectory=false; length=2036; replication=1; blocksize=4096; modification_time=1495797780000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
      FileStatus{path=ftp://test:123456@192.168.44.1/new/HADOOP-14486-001.patch; isDirectory=false; length=1322; replication=1; blocksize=4096; modification_time=1496716980000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
      FileStatus{path=ftp://test:123456@192.168.44.1/new/hadoop-main; isDirectory=true; modification_time=1495797120000; access_time=0; owner=user; group=group; permission=---------; isSymlink=false}
      

      In results above, FileStatus{path=ftp://test:123456@192.168.44.1/new; …… is obviously the current Path, and
      FileStatus{path=ftp://test:123456@192.168.44.1/;…… is obviously the parent Path.
      So, if we want to walk the directory recursively, it will stuck.

        Attachments

        1. HADOOP-14469-001.patch
          6 kB
          Hongyuan Li
        2. HADOOP-14469-002.patch
          6 kB
          Hongyuan Li
        3. HADOOP-14469-003.patch
          6 kB
          Hongyuan Li
        4. HADOOP-14469-004.patch
          6 kB
          Hongyuan Li
        5. HADOOP-14469-005.patch
          7 kB
          Hongyuan Li
        6. HADOOP-14469-006.patch
          10 kB
          Hongyuan Li
        7. HADOOP-14469-007.patch
          10 kB
          Hongyuan Li
        8. HADOOP-14469-008.patch
          10 kB
          Hongyuan Li

          Issue Links

            Activity

              People

              • Assignee:
                Hongyuan Li Hongyuan Li
                Reporter:
                Hongyuan Li Hongyuan Li
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: