[net] FTP Client fails to retrieve file list

• Key: NET-62
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Unassigned
• Reporter: Kevin Wilson
• Affects Version/s: 1.4

Environment

Operating System: Windows 2000
Platform: PC

Description

[
using commons-net-1.4.1.jar
jsdk 1.4.2_08
]

a) Both ftp servers require authorization.
b) login and authorization complete successfully.
c) change of working directory completes successfully.
d) At the end is the connection log using FileZilla to connect to ftp servers.
e) Both of the fetches below fail when a file exists in directory:

FTPListParseEngine engine = this.initiateListParsing();
FTPFile[] m_files = engine.getFiles();

OR

FTPFile[] m_files = this.listFiles();

where THIS = the org.apache.commons.net.ftp.FTPClient

Any ideas why I can't get a file list?

• • • • • • • • • • • FileZilla Connection Log ***************
                    • I replace server names/ips with <HOST (A or B)> ***********

Status: Connecting to <HOSTA> ...
Status: Connected with <HOSTA>. Waiting for welcome message...
Response: 220 ProFTPD 1.2.10 Server (<HOSTA>) [<HOSTA>]
Command: USER myusername
Response: 331 Password required for filer.
Command: PASS ******
Response: 230 User myusername logged in.
Command: FEAT
Response: 211-Features:
Response: MDTM
Response: REST STREAM
Response: SIZE
Response: 211 End
Command: SYST
Response: 215 UNIX Type: L8
Status: Connected
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is current directory.
Command: TYPE A
Response: 200 Type set to A
Command: PASV
Response: 227 Entering Passive Mode (<HOSTA IP LIST>).
Command: LIST
Response: 150 Opening ASCII mode data connection for file list
Response: 226 Transfer complete.
Status: Directory listing successful
Command: TYPE I
Response: 200 Type set to I
Command: PWD
Response: 257 "/" is current directory.
Command: REST 0
Response: 350 Restarting at 0. Send STORE or RETRIEVE to initiate transfer
Command: REST 0
Response: 350 Restarting at 0. Send STORE or RETRIEVE to initiate transfer
Status: Disconnected from server
Status: Connecting to <HOSTB> ...
Status: Connected with <HOSTB>. Waiting for welcome message...
Response: 220 <HOSTB> NcFTPd Server (licensed copy) ready.
Command: USER myusername
Response: 331 User myusername okay, need password.
Command: PASS **********
Response: 230-You are user #5 of 150 simultaneous users allowed.
Response: 230-
Response: 230 Restricted user logged in.
Command: FEAT
Response: 211-Extensions supported:
Response: CLNT
Response: LANG EN*
Response: MDTM
Response: MLST
Type*;Size*;Modify*;Perm;Unique;UNIX.mode*;UNIX.owner;UNIX.uid;UNIX.group;UNIX.gid;
Response: PASV
Response: REST STREAM
Response: SIZE
Response: UTF8
Response: TVFS
Response: Compliance Level: 20040701 (IETF mlst-16)
Response: 211 End.
Command: CLNT FileZilla
Response: 200 Noted.
Command: OPTS UTF8 ON
Response: 501 Option not recognized.
Command: SYST
Response: 215 UNIX Type: L8
Status: Connected
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is cwd.
Command: TYPE A
Response: 200 Type okay.
Command: PASV
Response: 227 Entering Passive Mode (<HOSTB IP List>)
Command: LIST
Response: 150 Data connection accepted from <CLIENT>:1229; transfer starting.
Response: 226 Listing completed.
Status: Directory listing successful
Command: TYPE A
Response: 200 Type okay.
Status: Disconnected from server

Comments

[Kevin Wilson]

Hold on, I think I have proxy issue that is screwin' stuff up.

[Kevin Wilson]

Ok,

I have verified all proxies are bypassed.

In issuing the raw FTPClient.list() command results in 2 codes being returned.

a) from our local ftp server I get CODE 425
b) from the remote ftp server I connect to I get CODE 550.

And no files are returned by using listFiles() still.

[Kevin Wilson]

Ok, I can fetch a file off our ftp server but I still can't get a directory
listing to check to see if any files are there to download.

[Kevin Wilson]

function FTPClient.retrieveFile() doesn't download ZIP files without corrupting
them, I can download a text file without issue. Changing the
FTPCLient.setFileTransferMode(mode) to binary before tranfer does not help.

[Kevin Wilson]

String[] FTPClient.listNames() works as expected. The cwd list of files and
directories are returned. There must be something wrong with the parser
detection or the building of the FTPFile object that doesn't allow listFiles()
to work properly.

[Kevin Wilson]

May have a problem that just might be the cause as to why listFiles() isn't
working. When attempting to use listFiles(path) to get a single file I get the
following exception. (code I am using is at the end of this message)

*****************************************************************************
java.lang.NoClassDefFoundError: org/apache/oro/text/regex/MalformedPatternException
at
org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory.createUnixFTPEntryParser(DefaultFTPFileEntryParserFactory.java:169)
at
org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory.createFileEntryParser(DefaultFTPFileEntryParserFactory.java:94)
at
org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:2358)
at org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2141)
at
com.comtrol.global.connections.FTPTransfer.checkFile(FTPTransfer.java:119)
at com.synfoserv.actions.ActionServlet.checkSPSFTP(ActionServlet.java:443)
at com.synfoserv.actions.ActionServlet.access$400(ActionServlet.java:27)
at com.synfoserv.actions.ActionServlet$2.run(ActionServlet.java:52)
at java.util.TimerThread.mainLoop(Timer.java:432)
at java.util.TimerThread.run(Timer.java:382)
*****************************************************************************

public FTPFile checkFile(String sServerFile){
FTPFile f = null;
FTPFile[] fa = null;
try{
fa = this.listFiles(sServerFile);
if(fa != null){ f = fa[0]; }
}catch(Exception e){System.out.println("checkFile() Error: \n" + e.getMessage());}
return f;
}

[Kevin Wilson]

RegexFTPFileEntryParserImpl.java still has these import definitions but the ORO
package is not included in src or jar file.

***********************************************************
import org.apache.oro.text.regex.MalformedPatternException;
import org.apache.oro.text.regex.MatchResult;
import org.apache.oro.text.regex.Pattern;
import org.apache.oro.text.regex.PatternMatcher;
import org.apache.oro.text.regex.Perl5Compiler;
import org.apache.oro.text.regex.Perl5Matcher;
***********************************************************

[Kevin Wilson]

VMSVersioningFTPEntryParser.java has the imports I mentioned as well.

[Kevin Wilson]

ORO package is required for commons-net-ftp parsers to function. I did not find
that mentioned anywhere on http://jakarta.apache.org/commons/net/ other than
stating the commons-net was originally developed by ORO, Inc.

Also, netbeans did not bark at all when the library was added, shouldn't it have?

[Kevin Wilson]

You must have the ORO package for the parser functions to work. It would be nice
for the commons-net download page to state that the ORO package is a dependency.

Zip files still download corrupted though. And so far FTPFile.isFile() is
returning false positive on directories.

[Steve Cohen]

Whoa. Sorry it's taken so long to respond (I was out of town) but you are
combining three or four issues into one defect report and that's never a good
idea. Please, in the future one bug report per defect. Now, let me try to help
you.

Re: the dependency on ORO: this has been published for years. It is listed on
the page http://jakarta.apache.org/commons/net/dependencies.html. A link to
that page is available under the menu heading "Project Info".

Re: corrupting zip files: I am guessing that this issue is the same as the one
you filed a few minutes later as a problem with WinZip files so I will not
comment on that here.

Based on that, I am closing the issue.

[Kevin Wilson]

Sorry but this page would be a better location to give a blurb about what else
is required to use the package (imo) ...

http://jakarta.apache.org/site/downloads/index.html

[Steve Cohen]

I see your point, but I don't agree that this page is the right one either. For
one thing that page is not under the control of commons-net developers but under
the control of the whole jakarta-commons system. I could imagine that a list of
dependencies there might get unwieldy.

At a bare minimum, though, I think this belongs at the very top of the FAQ, as
we hear this fairly frequently. Would you have looked there? Another idea is
that since the dependency is found only in a couple of places, we could trap the
ClassNotFoundException and emit a message that says explicitly that jakarta-oro
is required.

[Steve Cohen]

Reopened because I've decided to accomodate the reporter's request to make the
oro dependency more obvious.

[Steve Cohen]

Accomodated the desire for better documentation of the ORO dependency by
1) putting a question about it in the FAQ on the Wiki
2) wrapping the NoClassDefFoundError thrown in the parser creation process in a
ParserInitializationException with a message explicitly indicating the need for
jakarta-oro.jar to be on the classpath.

[Kevin Wilson]

Thanks, this should help immensely.

>2) wrapping the NoClassDefFoundError thrown in the parser creation process in a
>ParserInitializationException with a message explicitly indicating the need for
>jakarta-oro.jar to be on the classpath.