Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.1
    • Fix Version/s: 3.1
    • Component/s: FTP
    • Labels:
      None
    • Environment:

      Mac OS X Rumpus FTP Sevrer

      Description

      The Rumpus ftp server acts as an "old" pre mac os x ftp server.

      There are any plans do integrate a Mac OS filelist parsing into the commons net project?
      http://cyberduck.ch/ implemnets his own Rumpus parser,
      but it will be great if commons net already support this kind of listing.

        Activity

        Jürgen Jung created issue -
        Hide
        Sebb added a comment -

        If you can provide some sample listings - or, better, access to a server that runs Rumpus - then we can look at providing a parser if necessary.

        We also need the output from the SYST command, which is used to auto-detect the server type.

        Show
        Sebb added a comment - If you can provide some sample listings - or, better, access to a server that runs Rumpus - then we can look at providing a parser if necessary. We also need the output from the SYST command, which is used to auto-detect the server type.
        Hide
        Jürgen Jung added a comment -

        The Rumpus Server is available at http://www.maxum.com/Rumpus/

        The Standard SYST response is: "Mac OS Peter's Server"
        The FAQ describes why they use this response.

        I will attach an example listing this afternoon.

        Show
        Jürgen Jung added a comment - The Rumpus Server is available at http://www.maxum.com/Rumpus/ The Standard SYST response is: "Mac OS Peter's Server" The FAQ describes why they use this response. I will attach an example listing this afternoon.
        Hide
        Sebb added a comment -

        The Rumpus Server is available at http://www.maxum.com/Rumpus/

        However that requires installation on MacOs.

        Is there a public server running Rumpus?

        Show
        Sebb added a comment - The Rumpus Server is available at http://www.maxum.com/Rumpus/ However that requires installation on MacOs. Is there a public server running Rumpus?
        Hide
        Jürgen Jung added a comment -

        I don't know public rumpus server. :-/

        Show
        Jürgen Jung added a comment - I don't know public rumpus server. :-/
        Hide
        Jürgen Jung added a comment - - edited

        Standard login and listing of Rumpus FTP Server

        Server is installed on my local machine, ftp client is Transmit.

         
        Cmd: USER test
        331: Password required
        Cmd: PASS xxxxxxxx
        230: User logged in
        Cmd: TYPE A
        200: Type set to A (ASCII)
        Logged in to localhost as test.
        Cmd: SYST
        215: MACOS Peter's Server
        Cmd: FEAT
        211: Extensions supported:
              SIZE
              REST STREAM
              APPE
              MDTM
              MACB
              PBSZ
              PROT
              EPSV
              UTF8
             End
        Cmd: OPTS UTF8 ON
        220: UTF8 OPTS ON
        Cmd: PWD
        257: "/" is current directory.
        Cmd: PASV
        227: Entering Passive Mode (172,30,201,94,11,185)
        Fixing bogus PASV data address from 172.30.201.94:3001 to 127.0.0.1:3001.
        Cmd: LIST
        150: Opening connection
        226: Transfer complete
        -rw-r--r--    54149       27826    81975 Jul 22  2010 09.jpg
        drwxr-xr-x               folder        0 Jan  4 14:51 Alias_to_Steak
        -rw-r--r--    78440       49231   127671 Jul 22  2010 Filename with whitespace.jpg
        -rw-r--r--        0      108767   108767 Jul 22  2010 presentation03.jpg
        -rw-r--r--    58679       60393   119072 Jul 22  2010 presentation04.jpg
        -rw-r--r--    82543       51433   133976 Jul 22  2010 presentation06.jpg
        -rw-r--r--    83616     1430976  1514592 Jul 22  2010 presentation10.jpg
        -rw-r--r--        0       66990    66990 Jul 22  2010 presentation11.jpg
        drwxr-xr-x               folder        0 Jan  4 14:49 Steak
        -rwx------        0       12713    12713 Jul  8  2009 Twitter_Avatar.png
        

        Rumpus listing configuration is:

        For files:
        <mode> <rssz:8> <dtsz:11> <size:8> <date> <name>
        For folders:
        <mode>               folder        0 <date> <name>
        Show
        Jürgen Jung added a comment - - edited Standard login and listing of Rumpus FTP Server Server is installed on my local machine, ftp client is Transmit. Cmd: USER test 331: Password required Cmd: PASS xxxxxxxx 230: User logged in Cmd: TYPE A 200: Type set to A (ASCII) Logged in to localhost as test. Cmd: SYST 215: MACOS Peter's Server Cmd: FEAT 211: Extensions supported: SIZE REST STREAM APPE MDTM MACB PBSZ PROT EPSV UTF8 End Cmd: OPTS UTF8 ON 220: UTF8 OPTS ON Cmd: PWD 257: "/" is current directory. Cmd: PASV 227: Entering Passive Mode (172,30,201,94,11,185) Fixing bogus PASV data address from 172.30.201.94:3001 to 127.0.0.1:3001. Cmd: LIST 150: Opening connection 226: Transfer complete -rw-r--r-- 54149 27826 81975 Jul 22 2010 09.jpg drwxr-xr-x folder 0 Jan 4 14:51 Alias_to_Steak -rw-r--r-- 78440 49231 127671 Jul 22 2010 Filename with whitespace.jpg -rw-r--r-- 0 108767 108767 Jul 22 2010 presentation03.jpg -rw-r--r-- 58679 60393 119072 Jul 22 2010 presentation04.jpg -rw-r--r-- 82543 51433 133976 Jul 22 2010 presentation06.jpg -rw-r--r-- 83616 1430976 1514592 Jul 22 2010 presentation10.jpg -rw-r--r-- 0 66990 66990 Jul 22 2010 presentation11.jpg drwxr-xr-x folder 0 Jan 4 14:49 Steak -rwx------ 0 12713 12713 Jul 8 2009 Twitter_Avatar.png Rumpus listing configuration is: For files: <mode> <rssz:8> <dtsz:11> <size:8> <date> <name> For folders: <mode> folder 0 <date> <name>
        Show
        David Kocher added a comment - You are welcome to copy http://svn.cyberduck.ch/trunk/source/ch/cyberduck/core/ftp/parser/RumpusFTPEntryParser.java
        Hide
        Sebb added a comment -

        What happens when you try using FTPClient?

        For example:

        java -cp commons-net-examples-3.0.1.jar;commons-net-3.0.1.jar examples/ftp/FTPClientExample -l host user pass

        should list the directory contents; each file is listed twice, in raw and parsed versions, e.g.

        drwxr-xr-x 1 ukms ukms 1024 Jan 4 00:00 indexes
        drwxr-xr-x 1 ukms ukms 1024 2012-01-04 00:00:00 GMT indexes

        You can override the autodetection by using (for example):

        -Dorg.apache.commons.net.ftp.systemType=UNIX

        the value should be one of the FTPClientConfig constants (or a parser implementation).

        Show
        Sebb added a comment - What happens when you try using FTPClient? For example: java -cp commons-net-examples-3.0.1.jar;commons-net-3.0.1.jar examples/ftp/FTPClientExample -l host user pass should list the directory contents; each file is listed twice, in raw and parsed versions, e.g. drwxr-xr-x 1 ukms ukms 1024 Jan 4 00:00 indexes drwxr-xr-x 1 ukms ukms 1024 2012-01-04 00:00:00 GMT indexes You can override the autodetection by using (for example): -Dorg.apache.commons.net.ftp.systemType=UNIX the value should be one of the FTPClientConfig constants (or a parser implementation).
        Hide
        Jürgen Jung added a comment -

        What happens when you try using FTPClient?

        220-Welcome To Rumpus!
        220 Service ready for new user
        Connected to localhost on 21
        USER *******
        331 Password required
        PASS *******
        230 User logged in
        SYST
        215 MACOS Peter's Server
        Remote system is MACOS Peter's Server
        Exception in thread "main" org.apache.commons.net.ftp.parser.ParserInitializationException: Unknown parser type: MACOS Peter's Server
        	at org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory.createFileEntryParser(DefaultFTPFileEntryParserFactory.java:160)
        	at org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory.createFileEntryParser(DefaultFTPFileEntryParserFactory.java:93)
        	at org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:2959)
        	at org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2623)
        	at examples.ftp.FTPClientExample.main(FTPClientExample.java:299)
        

        With: -Dorg.apache.commons.net.ftp.systemType=UNIX

        1. All directories are missing
        2. files are listed as described
          20-Welcome To Rumpus!
          220 Service ready for new user
          Connected to localhost on 21
          USER *******
          331 Password required
          PASS *******
          230 User logged in
          SYST
          215 MACOS Peter's Server
          Remote system is MACOS Peter's Server
          PASV
          227 Entering Passive Mode (172,30,201,94,11,186)
          [Replacing site local address 172.30.201.94 with 127.0.0.1]
          LIST
          150 Opening connection
          226 Transfer complete
          -rw-r--r--    54149       27826    81975 Jul 22  2010 09.jpg
          -rw-r--r-- 54149 null     27826       81975 2010-07-22 00:00:00 MESZ  09.jpg
          -rw-r--r--    78440       49231   127671 Jul 22  2010 Filename with whitespace.jpg
          -rw-r--r-- 78440 null     49231      127671 2010-07-22 00:00:00 MESZ  Filename with whitespace.jpg
          -rw-r--r--        0      108767   108767 Jul 22  2010 presentation03.jpg
          -rw-r--r--    0 null     108767     108767 2010-07-22 00:00:00 MESZ  presentation03.jpg
          -rw-r--r--    58679       60393   119072 Jul 22  2010 presentation04.jpg
          -rw-r--r-- 58679 null     60393      119072 2010-07-22 00:00:00 MESZ  presentation04.jpg
          -rw-r--r--    82543       51433   133976 Jul 22  2010 presentation06.jpg
          -rw-r--r-- 82543 null     51433      133976 2010-07-22 00:00:00 MESZ  presentation06.jpg
          -rw-r--r--    83616     1430976  1514592 Jul 22  2010 presentation10.jpg
          -rw-r--r-- 83616 null     1430976   1514592 2010-07-22 00:00:00 MESZ  presentation10.jpg
          -rw-r--r--        0       66990    66990 Jul 22  2010 presentation11.jpg
          -rw-r--r--    0 null     66990       66990 2010-07-22 00:00:00 MESZ  presentation11.jpg
          -rwx------        0       12713    12713 Jul  8  2009 Twitter_Avatar.png
          -rwx------    0 null     12713       12713 2009-07-08 00:00:00 MESZ  Twitter_Avatar.png
          NOOP
          200 This space intentionally left blank <   >
          QUIT
          221-Goodbye From Rumpus!
          221 Goodbye - service closing connection
          
        Show
        Jürgen Jung added a comment - What happens when you try using FTPClient? 220-Welcome To Rumpus! 220 Service ready for new user Connected to localhost on 21 USER ******* 331 Password required PASS ******* 230 User logged in SYST 215 MACOS Peter's Server Remote system is MACOS Peter's Server Exception in thread "main" org.apache.commons.net.ftp.parser.ParserInitializationException: Unknown parser type: MACOS Peter's Server at org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory.createFileEntryParser(DefaultFTPFileEntryParserFactory.java:160) at org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory.createFileEntryParser(DefaultFTPFileEntryParserFactory.java:93) at org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:2959) at org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2623) at examples.ftp.FTPClientExample.main(FTPClientExample.java:299) With: -Dorg.apache.commons.net.ftp.systemType=UNIX All directories are missing files are listed as described 20-Welcome To Rumpus! 220 Service ready for new user Connected to localhost on 21 USER ******* 331 Password required PASS ******* 230 User logged in SYST 215 MACOS Peter's Server Remote system is MACOS Peter's Server PASV 227 Entering Passive Mode (172,30,201,94,11,186) [Replacing site local address 172.30.201.94 with 127.0.0.1] LIST 150 Opening connection 226 Transfer complete -rw-r--r-- 54149 27826 81975 Jul 22 2010 09.jpg -rw-r--r-- 54149 null 27826 81975 2010-07-22 00:00:00 MESZ 09.jpg -rw-r--r-- 78440 49231 127671 Jul 22 2010 Filename with whitespace.jpg -rw-r--r-- 78440 null 49231 127671 2010-07-22 00:00:00 MESZ Filename with whitespace.jpg -rw-r--r-- 0 108767 108767 Jul 22 2010 presentation03.jpg -rw-r--r-- 0 null 108767 108767 2010-07-22 00:00:00 MESZ presentation03.jpg -rw-r--r-- 58679 60393 119072 Jul 22 2010 presentation04.jpg -rw-r--r-- 58679 null 60393 119072 2010-07-22 00:00:00 MESZ presentation04.jpg -rw-r--r-- 82543 51433 133976 Jul 22 2010 presentation06.jpg -rw-r--r-- 82543 null 51433 133976 2010-07-22 00:00:00 MESZ presentation06.jpg -rw-r--r-- 83616 1430976 1514592 Jul 22 2010 presentation10.jpg -rw-r--r-- 83616 null 1430976 1514592 2010-07-22 00:00:00 MESZ presentation10.jpg -rw-r--r-- 0 66990 66990 Jul 22 2010 presentation11.jpg -rw-r--r-- 0 null 66990 66990 2010-07-22 00:00:00 MESZ presentation11.jpg -rwx------ 0 12713 12713 Jul 8 2009 Twitter_Avatar.png -rwx------ 0 null 12713 12713 2009-07-08 00:00:00 MESZ Twitter_Avatar.png NOOP 200 This space intentionally left blank < > QUIT 221-Goodbye From Rumpus! 221 Goodbye - service closing connection
        Hide
        Sebb added a comment -

        @Jurgen OK, thanks for the listing.

        What do rssz and dtsz represent?
        They appear to add up to size, so perhaps they are the resource fork size and data size?

        @David Thanks, will look at incorporating that.

        Show
        Sebb added a comment - @Jurgen OK, thanks for the listing. What do rssz and dtsz represent? They appear to add up to size, so perhaps they are the resource fork size and data size? @David Thanks, will look at incorporating that.
        Hide
        Jürgen Jung added a comment -

        What do rssz and dtsz represent?
        They appear to add up to size, so perhaps they are the resource fork size and data size?

        Correct:

        • rssz is the resource fork size
        • dtsz is the data size
        Show
        Jürgen Jung added a comment - What do rssz and dtsz represent? They appear to add up to size, so perhaps they are the resource fork size and data size? Correct: rssz is the resource fork size dtsz is the data size
        Hide
        Sebb added a comment -

        The Rumpus ftp server acts as an "old" pre mac os x ftp server.

        Does this mean that current Mac OS-X ftp servers act differently?
        If so, do they use a different SYST response?

        Show
        Sebb added a comment - The Rumpus ftp server acts as an "old" pre mac os x ftp server. Does this mean that current Mac OS-X ftp servers act differently? If so, do they use a different SYST response?
        Hide
        Jürgen Jung added a comment -

        If i enable the ftp server on my System (Mac OS X 10.7)

        SYST response is:

        215 UNIX Type: L8 Version: tnftpd 20100324+GSSAPI

        It acts like a normal Unix ftp server.

        Show
        Jürgen Jung added a comment - If i enable the ftp server on my System (Mac OS X 10.7) SYST response is: 215 UNIX Type: L8 Version: tnftpd 20100324+GSSAPI It acts like a normal Unix ftp server.
        Hide
        Sebb added a comment -

        OK, good, so if we check for MACOS Peter, we should be safe in assuming the "old" format.

        By the way, the Rumpus FAQ quotes the SYST response as

        Mac OS Peter's Server (heading, once)
        MacOS Peter's Server (body, multiple times)

        However, the actual response you quote is

        215 MACOS Peter's Server

        Show
        Sebb added a comment - OK, good, so if we check for MACOS Peter, we should be safe in assuming the "old" format. By the way, the Rumpus FAQ quotes the SYST response as Mac OS Peter's Server (heading, once) MacOS Peter's Server (body, multiple times) However, the actual response you quote is 215 MACOS Peter's Server
        Hide
        Jürgen Jung added a comment -

        Sounds good!

        Show
        Jürgen Jung added a comment - Sounds good!
        Hide
        Sebb added a comment -

        Fix added; perhaps you can try out the latest snapshot from https://repository.apache.org/content/repositories/snapshots/ ?

        Show
        Sebb added a comment - Fix added; perhaps you can try out the latest snapshot from https://repository.apache.org/content/repositories/snapshots/ ?
        Hide
        Jürgen Jung added a comment -

        Listing looks good except Encoding Problems on f***ing german umlauts.

        Expected listing:

        -rw-r--r--    58679       60393   119072 Jul 22  2010 Präˆsentation.jpg

        Example listing with auto detect encoding:

        -rw-r--r--    58679       60393   119072 Jul 22  2010 Präsentation.jpg

        Example with ftpClient.setControlEncoding("MacRoman");:

        -rw-r--r--    58679       60393   119072 Jul 22  2010 PraÃàsentation.jpg

        Example with ftpClient.setControlEncoding("MacRoman"); and string encoding new String(file.getRawListing().getBytes("MacRoman"),"UTF-8"):
        Works as expected:

        -rw-r--r--    58679       60393   119072 Jul 22  2010 Präsentation.jpg

        Example Client:

        public static void main(String[] args) throws Exception {
                FTPClient ftpClient = new FTPClient();
                ftpClient.setControlEncoding("MacRoman");
                ftpClient.connect("172.30.201.94");
                ftpClient.login("test", "test");
        
                FTPFile[] files = ftpClient.listFiles();
                for (FTPFile file : files) {
                    System.out.println(new String(file.getRawListing().getBytes("MacRoman"),"UTF-8"));
                    System.out.println(file.getRawListing());
                }
                ftpClient.disconnect();
        }
        
        Show
        Jürgen Jung added a comment - Listing looks good except Encoding Problems on f***ing german umlauts. Expected listing: -rw-r--r-- 58679 60393 119072 Jul 22 2010 Präˆsentation.jpg Example listing with auto detect encoding: -rw-r--r-- 58679 60393 119072 Jul 22 2010 Präsentation.jpg Example with ftpClient.setControlEncoding("MacRoman"); : -rw-r--r-- 58679 60393 119072 Jul 22 2010 PraÃàsentation.jpg Example with ftpClient.setControlEncoding("MacRoman"); and string encoding new String(file.getRawListing().getBytes("MacRoman"),"UTF-8") : Works as expected: -rw-r--r-- 58679 60393 119072 Jul 22 2010 Präsentation.jpg Example Client: public static void main( String [] args) throws Exception { FTPClient ftpClient = new FTPClient(); ftpClient.setControlEncoding( "MacRoman" ); ftpClient.connect( "172.30.201.94" ); ftpClient.login( "test" , "test" ); FTPFile[] files = ftpClient.listFiles(); for (FTPFile file : files) { System .out.println( new String (file.getRawListing().getBytes( "MacRoman" ), "UTF-8" )); System .out.println(file.getRawListing()); } ftpClient.disconnect(); }
        Hide
        Sebb added a comment -

        Are you saying that Net works with umlauts on your Mac when using the default ftp server and Unix parser, but does not work with Rumpus and the MacOsPeter parser?

        What happens with the Transmit client and Rumpus?

        Also, try using:

        ftpClient.setControlEncoding("UTF8");
        ...
        ftpClient.login...
        ftpClient.doCommand("OPTS","UTF8 ON");
        
        Show
        Sebb added a comment - Are you saying that Net works with umlauts on your Mac when using the default ftp server and Unix parser, but does not work with Rumpus and the MacOsPeter parser? What happens with the Transmit client and Rumpus? Also, try using: ftpClient.setControlEncoding( "UTF8" ); ... ftpClient.login... ftpClient.doCommand( "OPTS" , "UTF8 ON" );
        Hide
        Jürgen Jung added a comment - - edited

        Transmit shows the correct umlaut

        That works for rumpus and standard system ftp service:

        ftpClient.setControlEncoding("UTF8");
        ...
        ftpClient.login...
        ftpClient.doCommand("OPTS","UTF8 ON");
        
        Show
        Jürgen Jung added a comment - - edited Transmit shows the correct umlaut That works for rumpus and standard system ftp service: ftpClient.setControlEncoding("UTF8"); ... ftpClient.login... ftpClient.doCommand("OPTS","UTF8 ON");
        Hide
        Sebb added a comment -

        Transmit shows the correct umlaut

        That's presumably because Transmit sends the correct OPTS.

        That works for rumpus and standard system ftp service:

        So is the umlaut issue resolved?

        Show
        Sebb added a comment - Transmit shows the correct umlaut That's presumably because Transmit sends the correct OPTS. That works for rumpus and standard system ftp service: So is the umlaut issue resolved?
        Hide
        Jürgen Jung added a comment - - edited

        Yes when i send OPTS
        and use setControlEncoding
        the issue is resolved

        Show
        Jürgen Jung added a comment - - edited Yes when i send OPTS and use setControlEncoding the issue is resolved
        Hide
        Sebb added a comment -

        MacOS listing support now complete.

        Auto-detection of FTP server encoding might be worth raising as a separate enhancement issue; it probably applies to FTP servers on other systems as well.

        Show
        Sebb added a comment - MacOS listing support now complete. Auto-detection of FTP server encoding might be worth raising as a separate enhancement issue; it probably applies to FTP servers on other systems as well.
        Sebb made changes -
        Field Original Value New Value
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 3.1 [ 12317744 ]
        Resolution Fixed [ 1 ]
        Hide
        Jürgen Jung added a comment - - edited

        Great!
        Thanks a lot!

        Show
        Jürgen Jung added a comment - - edited Great! Thanks a lot!
        Sebb made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Jürgen Jung
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development