[NET-550] Default FTPClient bufferSize results in very slow retrieve transfers

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3
• Fix Version/s: 3.4
• Assignee: Unassigned
• Reporter: Geoffrey Hardy
• Created: 07/Aug/14 16:33
• Updated: 26/Nov/15 16:34
• Resolved: 03/Jul/15 16:40

Description

While experimenting with FTPClient, I discovered that if I don't call setBufferSize(), the default value is zero. This results in retrieveFile() calling the version of InputStream.read() with no parameters, reading one byte at a time. For comparison, the downloading a CD ISO image of about ~648MB took 18m10s with the default settings. In contrast, calling setBufferSize(8192) took only 7.9s, an improvement of ~137x.

Here is some sample code:

FTPClient ftp = new FTPClient();
// ftp.setBufferSize(8192);
ftp.setControlKeepAliveTimeout(300);
ftp.setCopyStreamListener(new CopyStreamListener() {
    @Override
    public void bytesTransferred(long totalBytesTransferred, int bytesTransferred, long streamSize) {
        System.out.println("totalBytesTransferred: " + totalBytesTransferred
            + ", bytesTransferred: " + bytesTransferred + ", streamSize: " + streamSize);
    }

    @Override public void bytesTransferred(CopyStreamEvent event) {}
});
ftp.connect(host);
ftp.login(user, pass);
ftp.retrieveFile(file, outputStream);

The log message from the stream listener printed lots messages "bytesTransferred: 1" and totalBytesTransferred incremented by 1 each time. This corresponds to the part of the code which reads one byte at a time with int inputStream.read().

Issue Links

• duplicates: NET-519 Apache Commons Net 3.3 has a performance issue
• relates to: NET-496 Util copyReader/copyStream classes should use default buffer size for non-positive buffer size parameters