Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-14947

camel-sftp: check for existance of remote directory using ls is very slow

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.25.1
    • 3.3.0
    • camel-ftp
    • None
    • Unknown

    Description

      In CAMEL-12733 the check for existing directories used by autoCreate was changed from ChannelSftp.cd to ChannelSftp.ls to fix a Windows specific issue.

      cd is a remote operation while ls will actually load the full directory listing and load it over the wire. This happens for every file that are to be written. If the remote directory has a lot of files this has a heavy performance penalty.

      In our case the directory some times contains 50.000+ files. Without autoCreate the writes take a few milliseconds. With autoCreate it takes up to 10 seconds due to the huge directory listing being generated and loaded over the wire for every single file to be written.

      This can be solved by reverting the directory check to a remote operation such as cd and fixing the Windows specific issue in another way.

      If it turns out that ls has to be used (unlikely) please note that there is another variant that takes a selector that enables you to abort the actual loading of the listing:

      ls(String path, LsEntrySelector selector)

       

      Attachments

        Issue Links

          Activity

            People

              davsclaus Claus Ibsen
              chrned Christian Nedregård
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: