Uploaded image for project: 'Commons Net'
  1. Commons Net
  2. NET-550

Default FTPClient bufferSize results in very slow retrieve transfers

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3
    • Fix Version/s: 3.4
    • Component/s: None
    • Labels:
      None

      Description

      While experimenting with FTPClient, I discovered that if I don't call setBufferSize(), the default value is zero. This results in retrieveFile() calling the version of InputStream.read() with no parameters, reading one byte at a time. For comparison, the downloading a CD ISO image of about ~648MB took 18m10s with the default settings. In contrast, calling setBufferSize(8192) took only 7.9s, an improvement of ~137x.

      Here is some sample code:

      FTPClient ftp = new FTPClient();
      // ftp.setBufferSize(8192);
      ftp.setControlKeepAliveTimeout(300);
      ftp.setCopyStreamListener(new CopyStreamListener() {
          @Override
          public void bytesTransferred(long totalBytesTransferred, int bytesTransferred, long streamSize) {
              System.out.println("totalBytesTransferred: " + totalBytesTransferred
                  + ", bytesTransferred: " + bytesTransferred + ", streamSize: " + streamSize);
          }
      
          @Override public void bytesTransferred(CopyStreamEvent event) {}
      });
      ftp.connect(host);
      ftp.login(user, pass);
      ftp.retrieveFile(file, outputStream);
      

      The log message from the stream listener printed lots messages "bytesTransferred: 1" and totalBytesTransferred incremented by 1 each time. This corresponds to the part of the code which reads one byte at a time with int inputStream.read().

        Attachments

        1. net-buffercopy.patch
          2 kB
          Bernd Eckenfels
        2. net-buffercopy-ext.patch
          9 kB
          Bernd Eckenfels

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                geoffhardy Geoffrey Hardy
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: